Codebase list libjuniversalchardet-java / fc8db10
Update upstream source from tag 'upstream/2.4.0' Update to upstream version '2.4.0' with Debian dir 59afb0e50b3c01e1b5d15d633ca162fae6f0de57 Markus Koschany 3 years ago
175 changed file(s) with 13066 addition(s) and 11073 deletion(s). Raw diff Collapse all Expand all
+71
-0
.github/workflows/codeql-analysis.yml less more
0 # For most projects, this workflow file will not need changing; you simply need
1 # to commit it to your repository.
2 #
3 # You may wish to alter this file to override the set of languages analyzed,
4 # or to provide custom queries or build logic.
5 name: "CodeQL"
6
7 on:
8 push:
9 branches: [master]
10 pull_request:
11 # The branches below must be a subset of the branches above
12 branches: [master]
13 schedule:
14 - cron: '0 13 * * 0'
15
16 jobs:
17 analyze:
18 name: Analyze
19 runs-on: ubuntu-latest
20
21 strategy:
22 fail-fast: false
23 matrix:
24 # Override automatic language detection by changing the below list
25 # Supported options are ['csharp', 'cpp', 'go', 'java', 'javascript', 'python']
26 language: ['java']
27 # Learn more...
28 # https://docs.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#overriding-automatic-language-detection
29
30 steps:
31 - name: Checkout repository
32 uses: actions/checkout@v2
33 with:
34 # We must fetch at least the immediate parents so that if this is
35 # a pull request then we can checkout the head.
36 fetch-depth: 2
37
38 # If this run was triggered by a pull request event, then checkout
39 # the head of the pull request instead of the merge commit.
40 - run: git checkout HEAD^2
41 if: ${{ github.event_name == 'pull_request' }}
42
43 # Initializes the CodeQL tools for scanning.
44 - name: Initialize CodeQL
45 uses: github/codeql-action/init@v1
46 with:
47 languages: ${{ matrix.language }}
48 # If you wish to specify custom queries, you can do so here or in a config file.
49 # By default, queries listed here will override any specified in a config file.
50 # Prefix the list here with "+" to use these queries and those in the config file.
51 # queries: ./path/to/local/query, your-org/your-repo/queries@main
52
53 # Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
54 # If this step fails, then you should remove it and run the build manually (see below)
55 - name: Autobuild
56 uses: github/codeql-action/autobuild@v1
57
58 # ℹ️ Command-line programs to run using the OS shell.
59 # 📚 https://git.io/JvXDl
60
61 # ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
62 # and modify them (or add more) to build your code if your project
63 # uses a compiled language
64
65 #- run: |
66 # make bootstrap
67 # make release
68
69 - name: Perform CodeQL Analysis
70 uses: github/codeql-action/analyze@v1
+5
-0
.gitignore less more
0 target/
1 .project
2 .classpath
3 .settings/
4 *~
+12
-0
.travis.yml less more
0 language: java
1
2 dist: trusty
3
4 install: mvn install -DskipTests=true -Dgpg.skip=true -Dmaven.javadoc.skip=true
5
6 jdk:
7 - openjdk7
8 - openjdk8
9 - openjdk9
10 - openjdk10
11 - openjdk11
+470
-0
LICENSE less more
0 MOZILLA PUBLIC LICENSE
1 Version 1.1
2
3 ---------------
4
5 1. Definitions.
6
7 1.0.1. "Commercial Use" means distribution or otherwise making the
8 Covered Code available to a third party.
9
10 1.1. "Contributor" means each entity that creates or contributes to
11 the creation of Modifications.
12
13 1.2. "Contributor Version" means the combination of the Original
14 Code, prior Modifications used by a Contributor, and the Modifications
15 made by that particular Contributor.
16
17 1.3. "Covered Code" means the Original Code or Modifications or the
18 combination of the Original Code and Modifications, in each case
19 including portions thereof.
20
21 1.4. "Electronic Distribution Mechanism" means a mechanism generally
22 accepted in the software development community for the electronic
23 transfer of data.
24
25 1.5. "Executable" means Covered Code in any form other than Source
26 Code.
27
28 1.6. "Initial Developer" means the individual or entity identified
29 as the Initial Developer in the Source Code notice required by Exhibit
30 A.
31
32 1.7. "Larger Work" means a work which combines Covered Code or
33 portions thereof with code not governed by the terms of this License.
34
35 1.8. "License" means this document.
36
37 1.8.1. "Licensable" means having the right to grant, to the maximum
38 extent possible, whether at the time of the initial grant or
39 subsequently acquired, any and all of the rights conveyed herein.
40
41 1.9. "Modifications" means any addition to or deletion from the
42 substance or structure of either the Original Code or any previous
43 Modifications. When Covered Code is released as a series of files, a
44 Modification is:
45 A. Any addition to or deletion from the contents of a file
46 containing Original Code or previous Modifications.
47
48 B. Any new file that contains any part of the Original Code or
49 previous Modifications.
50
51 1.10. "Original Code" means Source Code of computer software code
52 which is described in the Source Code notice required by Exhibit A as
53 Original Code, and which, at the time of its release under this
54 License is not already Covered Code governed by this License.
55
56 1.10.1. "Patent Claims" means any patent claim(s), now owned or
57 hereafter acquired, including without limitation, method, process,
58 and apparatus claims, in any patent Licensable by grantor.
59
60 1.11. "Source Code" means the preferred form of the Covered Code for
61 making modifications to it, including all modules it contains, plus
62 any associated interface definition files, scripts used to control
63 compilation and installation of an Executable, or source code
64 differential comparisons against either the Original Code or another
65 well known, available Covered Code of the Contributor's choice. The
66 Source Code can be in a compressed or archival form, provided the
67 appropriate decompression or de-archiving software is widely available
68 for no charge.
69
70 1.12. "You" (or "Your") means an individual or a legal entity
71 exercising rights under, and complying with all of the terms of, this
72 License or a future version of this License issued under Section 6.1.
73 For legal entities, "You" includes any entity which controls, is
74 controlled by, or is under common control with You. For purposes of
75 this definition, "control" means (a) the power, direct or indirect,
76 to cause the direction or management of such entity, whether by
77 contract or otherwise, or (b) ownership of more than fifty percent
78 (50%) of the outstanding shares or beneficial ownership of such
79 entity.
80
81 2. Source Code License.
82
83 2.1. The Initial Developer Grant.
84 The Initial Developer hereby grants You a world-wide, royalty-free,
85 non-exclusive license, subject to third party intellectual property
86 claims:
87 (a) under intellectual property rights (other than patent or
88 trademark) Licensable by Initial Developer to use, reproduce,
89 modify, display, perform, sublicense and distribute the Original
90 Code (or portions thereof) with or without Modifications, and/or
91 as part of a Larger Work; and
92
93 (b) under Patents Claims infringed by the making, using or
94 selling of Original Code, to make, have made, use, practice,
95 sell, and offer for sale, and/or otherwise dispose of the
96 Original Code (or portions thereof).
97
98 (c) the licenses granted in this Section 2.1(a) and (b) are
99 effective on the date Initial Developer first distributes
100 Original Code under the terms of this License.
101
102 (d) Notwithstanding Section 2.1(b) above, no patent license is
103 granted: 1) for code that You delete from the Original Code; 2)
104 separate from the Original Code; or 3) for infringements caused
105 by: i) the modification of the Original Code or ii) the
106 combination of the Original Code with other software or devices.
107
108 2.2. Contributor Grant.
109 Subject to third party intellectual property claims, each Contributor
110 hereby grants You a world-wide, royalty-free, non-exclusive license
111
112 (a) under intellectual property rights (other than patent or
113 trademark) Licensable by Contributor, to use, reproduce, modify,
114 display, perform, sublicense and distribute the Modifications
115 created by such Contributor (or portions thereof) either on an
116 unmodified basis, with other Modifications, as Covered Code
117 and/or as part of a Larger Work; and
118
119 (b) under Patent Claims infringed by the making, using, or
120 selling of Modifications made by that Contributor either alone
121 and/or in combination with its Contributor Version (or portions
122 of such combination), to make, use, sell, offer for sale, have
123 made, and/or otherwise dispose of: 1) Modifications made by that
124 Contributor (or portions thereof); and 2) the combination of
125 Modifications made by that Contributor with its Contributor
126 Version (or portions of such combination).
127
128 (c) the licenses granted in Sections 2.2(a) and 2.2(b) are
129 effective on the date Contributor first makes Commercial Use of
130 the Covered Code.
131
132 (d) Notwithstanding Section 2.2(b) above, no patent license is
133 granted: 1) for any code that Contributor has deleted from the
134 Contributor Version; 2) separate from the Contributor Version;
135 3) for infringements caused by: i) third party modifications of
136 Contributor Version or ii) the combination of Modifications made
137 by that Contributor with other software (except as part of the
138 Contributor Version) or other devices; or 4) under Patent Claims
139 infringed by Covered Code in the absence of Modifications made by
140 that Contributor.
141
142 3. Distribution Obligations.
143
144 3.1. Application of License.
145 The Modifications which You create or to which You contribute are
146 governed by the terms of this License, including without limitation
147 Section 2.2. The Source Code version of Covered Code may be
148 distributed only under the terms of this License or a future version
149 of this License released under Section 6.1, and You must include a
150 copy of this License with every copy of the Source Code You
151 distribute. You may not offer or impose any terms on any Source Code
152 version that alters or restricts the applicable version of this
153 License or the recipients' rights hereunder. However, You may include
154 an additional document offering the additional rights described in
155 Section 3.5.
156
157 3.2. Availability of Source Code.
158 Any Modification which You create or to which You contribute must be
159 made available in Source Code form under the terms of this License
160 either on the same media as an Executable version or via an accepted
161 Electronic Distribution Mechanism to anyone to whom you made an
162 Executable version available; and if made available via Electronic
163 Distribution Mechanism, must remain available for at least twelve (12)
164 months after the date it initially became available, or at least six
165 (6) months after a subsequent version of that particular Modification
166 has been made available to such recipients. You are responsible for
167 ensuring that the Source Code version remains available even if the
168 Electronic Distribution Mechanism is maintained by a third party.
169
170 3.3. Description of Modifications.
171 You must cause all Covered Code to which You contribute to contain a
172 file documenting the changes You made to create that Covered Code and
173 the date of any change. You must include a prominent statement that
174 the Modification is derived, directly or indirectly, from Original
175 Code provided by the Initial Developer and including the name of the
176 Initial Developer in (a) the Source Code, and (b) in any notice in an
177 Executable version or related documentation in which You describe the
178 origin or ownership of the Covered Code.
179
180 3.4. Intellectual Property Matters
181 (a) Third Party Claims.
182 If Contributor has knowledge that a license under a third party's
183 intellectual property rights is required to exercise the rights
184 granted by such Contributor under Sections 2.1 or 2.2,
185 Contributor must include a text file with the Source Code
186 distribution titled "LEGAL" which describes the claim and the
187 party making the claim in sufficient detail that a recipient will
188 know whom to contact. If Contributor obtains such knowledge after
189 the Modification is made available as described in Section 3.2,
190 Contributor shall promptly modify the LEGAL file in all copies
191 Contributor makes available thereafter and shall take other steps
192 (such as notifying appropriate mailing lists or newsgroups)
193 reasonably calculated to inform those who received the Covered
194 Code that new knowledge has been obtained.
195
196 (b) Contributor APIs.
197 If Contributor's Modifications include an application programming
198 interface and Contributor has knowledge of patent licenses which
199 are reasonably necessary to implement that API, Contributor must
200 also include this information in the LEGAL file.
201
202 (c) Representations.
203 Contributor represents that, except as disclosed pursuant to
204 Section 3.4(a) above, Contributor believes that Contributor's
205 Modifications are Contributor's original creation(s) and/or
206 Contributor has sufficient rights to grant the rights conveyed by
207 this License.
208
209 3.5. Required Notices.
210 You must duplicate the notice in Exhibit A in each file of the Source
211 Code. If it is not possible to put such notice in a particular Source
212 Code file due to its structure, then You must include such notice in a
213 location (such as a relevant directory) where a user would be likely
214 to look for such a notice. If You created one or more Modification(s)
215 You may add your name as a Contributor to the notice described in
216 Exhibit A. You must also duplicate this License in any documentation
217 for the Source Code where You describe recipients' rights or ownership
218 rights relating to Covered Code. You may choose to offer, and to
219 charge a fee for, warranty, support, indemnity or liability
220 obligations to one or more recipients of Covered Code. However, You
221 may do so only on Your own behalf, and not on behalf of the Initial
222 Developer or any Contributor. You must make it absolutely clear than
223 any such warranty, support, indemnity or liability obligation is
224 offered by You alone, and You hereby agree to indemnify the Initial
225 Developer and every Contributor for any liability incurred by the
226 Initial Developer or such Contributor as a result of warranty,
227 support, indemnity or liability terms You offer.
228
229 3.6. Distribution of Executable Versions.
230 You may distribute Covered Code in Executable form only if the
231 requirements of Section 3.1-3.5 have been met for that Covered Code,
232 and if You include a notice stating that the Source Code version of
233 the Covered Code is available under the terms of this License,
234 including a description of how and where You have fulfilled the
235 obligations of Section 3.2. The notice must be conspicuously included
236 in any notice in an Executable version, related documentation or
237 collateral in which You describe recipients' rights relating to the
238 Covered Code. You may distribute the Executable version of Covered
239 Code or ownership rights under a license of Your choice, which may
240 contain terms different from this License, provided that You are in
241 compliance with the terms of this License and that the license for the
242 Executable version does not attempt to limit or alter the recipient's
243 rights in the Source Code version from the rights set forth in this
244 License. If You distribute the Executable version under a different
245 license You must make it absolutely clear that any terms which differ
246 from this License are offered by You alone, not by the Initial
247 Developer or any Contributor. You hereby agree to indemnify the
248 Initial Developer and every Contributor for any liability incurred by
249 the Initial Developer or such Contributor as a result of any such
250 terms You offer.
251
252 3.7. Larger Works.
253 You may create a Larger Work by combining Covered Code with other code
254 not governed by the terms of this License and distribute the Larger
255 Work as a single product. In such a case, You must make sure the
256 requirements of this License are fulfilled for the Covered Code.
257
258 4. Inability to Comply Due to Statute or Regulation.
259
260 If it is impossible for You to comply with any of the terms of this
261 License with respect to some or all of the Covered Code due to
262 statute, judicial order, or regulation then You must: (a) comply with
263 the terms of this License to the maximum extent possible; and (b)
264 describe the limitations and the code they affect. Such description
265 must be included in the LEGAL file described in Section 3.4 and must
266 be included with all distributions of the Source Code. Except to the
267 extent prohibited by statute or regulation, such description must be
268 sufficiently detailed for a recipient of ordinary skill to be able to
269 understand it.
270
271 5. Application of this License.
272
273 This License applies to code to which the Initial Developer has
274 attached the notice in Exhibit A and to related Covered Code.
275
276 6. Versions of the License.
277
278 6.1. New Versions.
279 Netscape Communications Corporation ("Netscape") may publish revised
280 and/or new versions of the License from time to time. Each version
281 will be given a distinguishing version number.
282
283 6.2. Effect of New Versions.
284 Once Covered Code has been published under a particular version of the
285 License, You may always continue to use it under the terms of that
286 version. You may also choose to use such Covered Code under the terms
287 of any subsequent version of the License published by Netscape. No one
288 other than Netscape has the right to modify the terms applicable to
289 Covered Code created under this License.
290
291 6.3. Derivative Works.
292 If You create or use a modified version of this License (which you may
293 only do in order to apply it to code which is not already Covered Code
294 governed by this License), You must (a) rename Your license so that
295 the phrases "Mozilla", "MOZILLAPL", "MOZPL", "Netscape",
296 "MPL", "NPL" or any confusingly similar phrase do not appear in your
297 license (except to note that your license differs from this License)
298 and (b) otherwise make it clear that Your version of the license
299 contains terms which differ from the Mozilla Public License and
300 Netscape Public License. (Filling in the name of the Initial
301 Developer, Original Code or Contributor in the notice described in
302 Exhibit A shall not of themselves be deemed to be modifications of
303 this License.)
304
305 7. DISCLAIMER OF WARRANTY.
306
307 COVERED CODE IS PROVIDED UNDER THIS LICENSE ON AN "AS IS" BASIS,
308 WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING,
309 WITHOUT LIMITATION, WARRANTIES THAT THE COVERED CODE IS FREE OF
310 DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR PURPOSE OR NON-INFRINGING.
311 THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE COVERED CODE
312 IS WITH YOU. SHOULD ANY COVERED CODE PROVE DEFECTIVE IN ANY RESPECT,
313 YOU (NOT THE INITIAL DEVELOPER OR ANY OTHER CONTRIBUTOR) ASSUME THE
314 COST OF ANY NECESSARY SERVICING, REPAIR OR CORRECTION. THIS DISCLAIMER
315 OF WARRANTY CONSTITUTES AN ESSENTIAL PART OF THIS LICENSE. NO USE OF
316 ANY COVERED CODE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS DISCLAIMER.
317
318 8. TERMINATION.
319
320 8.1. This License and the rights granted hereunder will terminate
321 automatically if You fail to comply with terms herein and fail to cure
322 such breach within 30 days of becoming aware of the breach. All
323 sublicenses to the Covered Code which are properly granted shall
324 survive any termination of this License. Provisions which, by their
325 nature, must remain in effect beyond the termination of this License
326 shall survive.
327
328 8.2. If You initiate litigation by asserting a patent infringement
329 claim (excluding declatory judgment actions) against Initial Developer
330 or a Contributor (the Initial Developer or Contributor against whom
331 You file such action is referred to as "Participant") alleging that:
332
333 (a) such Participant's Contributor Version directly or indirectly
334 infringes any patent, then any and all rights granted by such
335 Participant to You under Sections 2.1 and/or 2.2 of this License
336 shall, upon 60 days notice from Participant terminate prospectively,
337 unless if within 60 days after receipt of notice You either: (i)
338 agree in writing to pay Participant a mutually agreeable reasonable
339 royalty for Your past and future use of Modifications made by such
340 Participant, or (ii) withdraw Your litigation claim with respect to
341 the Contributor Version against such Participant. If within 60 days
342 of notice, a reasonable royalty and payment arrangement are not
343 mutually agreed upon in writing by the parties or the litigation claim
344 is not withdrawn, the rights granted by Participant to You under
345 Sections 2.1 and/or 2.2 automatically terminate at the expiration of
346 the 60 day notice period specified above.
347
348 (b) any software, hardware, or device, other than such Participant's
349 Contributor Version, directly or indirectly infringes any patent, then
350 any rights granted to You by such Participant under Sections 2.1(b)
351 and 2.2(b) are revoked effective as of the date You first made, used,
352 sold, distributed, or had made, Modifications made by that
353 Participant.
354
355 8.3. If You assert a patent infringement claim against Participant
356 alleging that such Participant's Contributor Version directly or
357 indirectly infringes any patent where such claim is resolved (such as
358 by license or settlement) prior to the initiation of patent
359 infringement litigation, then the reasonable value of the licenses
360 granted by such Participant under Sections 2.1 or 2.2 shall be taken
361 into account in determining the amount or value of any payment or
362 license.
363
364 8.4. In the event of termination under Sections 8.1 or 8.2 above,
365 all end user license agreements (excluding distributors and resellers)
366 which have been validly granted by You or any distributor hereunder
367 prior to termination shall survive termination.
368
369 9. LIMITATION OF LIABILITY.
370
371 UNDER NO CIRCUMSTANCES AND UNDER NO LEGAL THEORY, WHETHER TORT
372 (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE, SHALL YOU, THE INITIAL
373 DEVELOPER, ANY OTHER CONTRIBUTOR, OR ANY DISTRIBUTOR OF COVERED CODE,
374 OR ANY SUPPLIER OF ANY OF SUCH PARTIES, BE LIABLE TO ANY PERSON FOR
375 ANY INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES OF ANY
376 CHARACTER INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF GOODWILL,
377 WORK STOPPAGE, COMPUTER FAILURE OR MALFUNCTION, OR ANY AND ALL OTHER
378 COMMERCIAL DAMAGES OR LOSSES, EVEN IF SUCH PARTY SHALL HAVE BEEN
379 INFORMED OF THE POSSIBILITY OF SUCH DAMAGES. THIS LIMITATION OF
380 LIABILITY SHALL NOT APPLY TO LIABILITY FOR DEATH OR PERSONAL INJURY
381 RESULTING FROM SUCH PARTY'S NEGLIGENCE TO THE EXTENT APPLICABLE LAW
382 PROHIBITS SUCH LIMITATION. SOME JURISDICTIONS DO NOT ALLOW THE
383 EXCLUSION OR LIMITATION OF INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO
384 THIS EXCLUSION AND LIMITATION MAY NOT APPLY TO YOU.
385
386 10. U.S. GOVERNMENT END USERS.
387
388 The Covered Code is a "commercial item," as that term is defined in
389 48 C.F.R. 2.101 (Oct. 1995), consisting of "commercial computer
390 software" and "commercial computer software documentation," as such
391 terms are used in 48 C.F.R. 12.212 (Sept. 1995). Consistent with 48
392 C.F.R. 12.212 and 48 C.F.R. 227.7202-1 through 227.7202-4 (June 1995),
393 all U.S. Government End Users acquire Covered Code with only those
394 rights set forth herein.
395
396 11. MISCELLANEOUS.
397
398 This License represents the complete agreement concerning subject
399 matter hereof. If any provision of this License is held to be
400 unenforceable, such provision shall be reformed only to the extent
401 necessary to make it enforceable. This License shall be governed by
402 California law provisions (except to the extent applicable law, if
403 any, provides otherwise), excluding its conflict-of-law provisions.
404 With respect to disputes in which at least one party is a citizen of,
405 or an entity chartered or registered to do business in the United
406 States of America, any litigation relating to this License shall be
407 subject to the jurisdiction of the Federal Courts of the Northern
408 District of California, with venue lying in Santa Clara County,
409 California, with the losing party responsible for costs, including
410 without limitation, court costs and reasonable attorneys' fees and
411 expenses. The application of the United Nations Convention on
412 Contracts for the International Sale of Goods is expressly excluded.
413 Any law or regulation which provides that the language of a contract
414 shall be construed against the drafter shall not apply to this
415 License.
416
417 12. RESPONSIBILITY FOR CLAIMS.
418
419 As between Initial Developer and the Contributors, each party is
420 responsible for claims and damages arising, directly or indirectly,
421 out of its utilization of rights under this License and You agree to
422 work with Initial Developer and Contributors to distribute such
423 responsibility on an equitable basis. Nothing herein is intended or
424 shall be deemed to constitute any admission of liability.
425
426 13. MULTIPLE-LICENSED CODE.
427
428 Initial Developer may designate portions of the Covered Code as
429 "Multiple-Licensed". "Multiple-Licensed" means that the Initial
430 Developer permits you to utilize portions of the Covered Code under
431 Your choice of the NPL or the alternative licenses, if any, specified
432 by the Initial Developer in the file described in Exhibit A.
433
434 EXHIBIT A -Mozilla Public License.
435
436 ``The contents of this file are subject to the Mozilla Public License
437 Version 1.1 (the "License"); you may not use this file except in
438 compliance with the License. You may obtain a copy of the License at
439 http://www.mozilla.org/MPL/
440
441 Software distributed under the License is distributed on an "AS IS"
442 basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
443 License for the specific language governing rights and limitations
444 under the License.
445
446 The Original Code is ______________________________________.
447
448 The Initial Developer of the Original Code is ________________________.
449 Portions created by ______________________ are Copyright (C) ______
450 _______________________. All Rights Reserved.
451
452 Contributor(s): ______________________________________.
453
454 Alternatively, the contents of this file may be used under the terms
455 of the _____ license (the "[___] License"), in which case the
456 provisions of [______] License are applicable instead of those
457 above. If you wish to allow use of your version of this file only
458 under the terms of the [____] License and not to allow others to use
459 your version of this file under the MPL, indicate your decision by
460 deleting the provisions above and replace them with the notice and
461 other provisions required by the [___] License. If you do not delete
462 the provisions above, a recipient may use your version of this file
463 under either the MPL or the [___] License."
464
465 [NOTE: The text of this Exhibit A may differ slightly from the text of
466 the notices in the Source Code files of the Original Code. You should
467 use the text of this Exhibit A rather than the text found in the
468 Original Code Source Code for Your Modifications.]
469
+0
-470
MPL-1.1.txt less more
0 MOZILLA PUBLIC LICENSE
1 Version 1.1
2
3 ---------------
4
5 1. Definitions.
6
7 1.0.1. "Commercial Use" means distribution or otherwise making the
8 Covered Code available to a third party.
9
10 1.1. "Contributor" means each entity that creates or contributes to
11 the creation of Modifications.
12
13 1.2. "Contributor Version" means the combination of the Original
14 Code, prior Modifications used by a Contributor, and the Modifications
15 made by that particular Contributor.
16
17 1.3. "Covered Code" means the Original Code or Modifications or the
18 combination of the Original Code and Modifications, in each case
19 including portions thereof.
20
21 1.4. "Electronic Distribution Mechanism" means a mechanism generally
22 accepted in the software development community for the electronic
23 transfer of data.
24
25 1.5. "Executable" means Covered Code in any form other than Source
26 Code.
27
28 1.6. "Initial Developer" means the individual or entity identified
29 as the Initial Developer in the Source Code notice required by Exhibit
30 A.
31
32 1.7. "Larger Work" means a work which combines Covered Code or
33 portions thereof with code not governed by the terms of this License.
34
35 1.8. "License" means this document.
36
37 1.8.1. "Licensable" means having the right to grant, to the maximum
38 extent possible, whether at the time of the initial grant or
39 subsequently acquired, any and all of the rights conveyed herein.
40
41 1.9. "Modifications" means any addition to or deletion from the
42 substance or structure of either the Original Code or any previous
43 Modifications. When Covered Code is released as a series of files, a
44 Modification is:
45 A. Any addition to or deletion from the contents of a file
46 containing Original Code or previous Modifications.
47
48 B. Any new file that contains any part of the Original Code or
49 previous Modifications.
50
51 1.10. "Original Code" means Source Code of computer software code
52 which is described in the Source Code notice required by Exhibit A as
53 Original Code, and which, at the time of its release under this
54 License is not already Covered Code governed by this License.
55
56 1.10.1. "Patent Claims" means any patent claim(s), now owned or
57 hereafter acquired, including without limitation, method, process,
58 and apparatus claims, in any patent Licensable by grantor.
59
60 1.11. "Source Code" means the preferred form of the Covered Code for
61 making modifications to it, including all modules it contains, plus
62 any associated interface definition files, scripts used to control
63 compilation and installation of an Executable, or source code
64 differential comparisons against either the Original Code or another
65 well known, available Covered Code of the Contributor's choice. The
66 Source Code can be in a compressed or archival form, provided the
67 appropriate decompression or de-archiving software is widely available
68 for no charge.
69
70 1.12. "You" (or "Your") means an individual or a legal entity
71 exercising rights under, and complying with all of the terms of, this
72 License or a future version of this License issued under Section 6.1.
73 For legal entities, "You" includes any entity which controls, is
74 controlled by, or is under common control with You. For purposes of
75 this definition, "control" means (a) the power, direct or indirect,
76 to cause the direction or management of such entity, whether by
77 contract or otherwise, or (b) ownership of more than fifty percent
78 (50%) of the outstanding shares or beneficial ownership of such
79 entity.
80
81 2. Source Code License.
82
83 2.1. The Initial Developer Grant.
84 The Initial Developer hereby grants You a world-wide, royalty-free,
85 non-exclusive license, subject to third party intellectual property
86 claims:
87 (a) under intellectual property rights (other than patent or
88 trademark) Licensable by Initial Developer to use, reproduce,
89 modify, display, perform, sublicense and distribute the Original
90 Code (or portions thereof) with or without Modifications, and/or
91 as part of a Larger Work; and
92
93 (b) under Patents Claims infringed by the making, using or
94 selling of Original Code, to make, have made, use, practice,
95 sell, and offer for sale, and/or otherwise dispose of the
96 Original Code (or portions thereof).
97
98 (c) the licenses granted in this Section 2.1(a) and (b) are
99 effective on the date Initial Developer first distributes
100 Original Code under the terms of this License.
101
102 (d) Notwithstanding Section 2.1(b) above, no patent license is
103 granted: 1) for code that You delete from the Original Code; 2)
104 separate from the Original Code; or 3) for infringements caused
105 by: i) the modification of the Original Code or ii) the
106 combination of the Original Code with other software or devices.
107
108 2.2. Contributor Grant.
109 Subject to third party intellectual property claims, each Contributor
110 hereby grants You a world-wide, royalty-free, non-exclusive license
111
112 (a) under intellectual property rights (other than patent or
113 trademark) Licensable by Contributor, to use, reproduce, modify,
114 display, perform, sublicense and distribute the Modifications
115 created by such Contributor (or portions thereof) either on an
116 unmodified basis, with other Modifications, as Covered Code
117 and/or as part of a Larger Work; and
118
119 (b) under Patent Claims infringed by the making, using, or
120 selling of Modifications made by that Contributor either alone
121 and/or in combination with its Contributor Version (or portions
122 of such combination), to make, use, sell, offer for sale, have
123 made, and/or otherwise dispose of: 1) Modifications made by that
124 Contributor (or portions thereof); and 2) the combination of
125 Modifications made by that Contributor with its Contributor
126 Version (or portions of such combination).
127
128 (c) the licenses granted in Sections 2.2(a) and 2.2(b) are
129 effective on the date Contributor first makes Commercial Use of
130 the Covered Code.
131
132 (d) Notwithstanding Section 2.2(b) above, no patent license is
133 granted: 1) for any code that Contributor has deleted from the
134 Contributor Version; 2) separate from the Contributor Version;
135 3) for infringements caused by: i) third party modifications of
136 Contributor Version or ii) the combination of Modifications made
137 by that Contributor with other software (except as part of the
138 Contributor Version) or other devices; or 4) under Patent Claims
139 infringed by Covered Code in the absence of Modifications made by
140 that Contributor.
141
142 3. Distribution Obligations.
143
144 3.1. Application of License.
145 The Modifications which You create or to which You contribute are
146 governed by the terms of this License, including without limitation
147 Section 2.2. The Source Code version of Covered Code may be
148 distributed only under the terms of this License or a future version
149 of this License released under Section 6.1, and You must include a
150 copy of this License with every copy of the Source Code You
151 distribute. You may not offer or impose any terms on any Source Code
152 version that alters or restricts the applicable version of this
153 License or the recipients' rights hereunder. However, You may include
154 an additional document offering the additional rights described in
155 Section 3.5.
156
157 3.2. Availability of Source Code.
158 Any Modification which You create or to which You contribute must be
159 made available in Source Code form under the terms of this License
160 either on the same media as an Executable version or via an accepted
161 Electronic Distribution Mechanism to anyone to whom you made an
162 Executable version available; and if made available via Electronic
163 Distribution Mechanism, must remain available for at least twelve (12)
164 months after the date it initially became available, or at least six
165 (6) months after a subsequent version of that particular Modification
166 has been made available to such recipients. You are responsible for
167 ensuring that the Source Code version remains available even if the
168 Electronic Distribution Mechanism is maintained by a third party.
169
170 3.3. Description of Modifications.
171 You must cause all Covered Code to which You contribute to contain a
172 file documenting the changes You made to create that Covered Code and
173 the date of any change. You must include a prominent statement that
174 the Modification is derived, directly or indirectly, from Original
175 Code provided by the Initial Developer and including the name of the
176 Initial Developer in (a) the Source Code, and (b) in any notice in an
177 Executable version or related documentation in which You describe the
178 origin or ownership of the Covered Code.
179
180 3.4. Intellectual Property Matters
181 (a) Third Party Claims.
182 If Contributor has knowledge that a license under a third party's
183 intellectual property rights is required to exercise the rights
184 granted by such Contributor under Sections 2.1 or 2.2,
185 Contributor must include a text file with the Source Code
186 distribution titled "LEGAL" which describes the claim and the
187 party making the claim in sufficient detail that a recipient will
188 know whom to contact. If Contributor obtains such knowledge after
189 the Modification is made available as described in Section 3.2,
190 Contributor shall promptly modify the LEGAL file in all copies
191 Contributor makes available thereafter and shall take other steps
192 (such as notifying appropriate mailing lists or newsgroups)
193 reasonably calculated to inform those who received the Covered
194 Code that new knowledge has been obtained.
195
196 (b) Contributor APIs.
197 If Contributor's Modifications include an application programming
198 interface and Contributor has knowledge of patent licenses which
199 are reasonably necessary to implement that API, Contributor must
200 also include this information in the LEGAL file.
201
202 (c) Representations.
203 Contributor represents that, except as disclosed pursuant to
204 Section 3.4(a) above, Contributor believes that Contributor's
205 Modifications are Contributor's original creation(s) and/or
206 Contributor has sufficient rights to grant the rights conveyed by
207 this License.
208
209 3.5. Required Notices.
210 You must duplicate the notice in Exhibit A in each file of the Source
211 Code. If it is not possible to put such notice in a particular Source
212 Code file due to its structure, then You must include such notice in a
213 location (such as a relevant directory) where a user would be likely
214 to look for such a notice. If You created one or more Modification(s)
215 You may add your name as a Contributor to the notice described in
216 Exhibit A. You must also duplicate this License in any documentation
217 for the Source Code where You describe recipients' rights or ownership
218 rights relating to Covered Code. You may choose to offer, and to
219 charge a fee for, warranty, support, indemnity or liability
220 obligations to one or more recipients of Covered Code. However, You
221 may do so only on Your own behalf, and not on behalf of the Initial
222 Developer or any Contributor. You must make it absolutely clear than
223 any such warranty, support, indemnity or liability obligation is
224 offered by You alone, and You hereby agree to indemnify the Initial
225 Developer and every Contributor for any liability incurred by the
226 Initial Developer or such Contributor as a result of warranty,
227 support, indemnity or liability terms You offer.
228
229 3.6. Distribution of Executable Versions.
230 You may distribute Covered Code in Executable form only if the
231 requirements of Section 3.1-3.5 have been met for that Covered Code,
232 and if You include a notice stating that the Source Code version of
233 the Covered Code is available under the terms of this License,
234 including a description of how and where You have fulfilled the
235 obligations of Section 3.2. The notice must be conspicuously included
236 in any notice in an Executable version, related documentation or
237 collateral in which You describe recipients' rights relating to the
238 Covered Code. You may distribute the Executable version of Covered
239 Code or ownership rights under a license of Your choice, which may
240 contain terms different from this License, provided that You are in
241 compliance with the terms of this License and that the license for the
242 Executable version does not attempt to limit or alter the recipient's
243 rights in the Source Code version from the rights set forth in this
244 License. If You distribute the Executable version under a different
245 license You must make it absolutely clear that any terms which differ
246 from this License are offered by You alone, not by the Initial
247 Developer or any Contributor. You hereby agree to indemnify the
248 Initial Developer and every Contributor for any liability incurred by
249 the Initial Developer or such Contributor as a result of any such
250 terms You offer.
251
252 3.7. Larger Works.
253 You may create a Larger Work by combining Covered Code with other code
254 not governed by the terms of this License and distribute the Larger
255 Work as a single product. In such a case, You must make sure the
256 requirements of this License are fulfilled for the Covered Code.
257
258 4. Inability to Comply Due to Statute or Regulation.
259
260 If it is impossible for You to comply with any of the terms of this
261 License with respect to some or all of the Covered Code due to
262 statute, judicial order, or regulation then You must: (a) comply with
263 the terms of this License to the maximum extent possible; and (b)
264 describe the limitations and the code they affect. Such description
265 must be included in the LEGAL file described in Section 3.4 and must
266 be included with all distributions of the Source Code. Except to the
267 extent prohibited by statute or regulation, such description must be
268 sufficiently detailed for a recipient of ordinary skill to be able to
269 understand it.
270
271 5. Application of this License.
272
273 This License applies to code to which the Initial Developer has
274 attached the notice in Exhibit A and to related Covered Code.
275
276 6. Versions of the License.
277
278 6.1. New Versions.
279 Netscape Communications Corporation ("Netscape") may publish revised
280 and/or new versions of the License from time to time. Each version
281 will be given a distinguishing version number.
282
283 6.2. Effect of New Versions.
284 Once Covered Code has been published under a particular version of the
285 License, You may always continue to use it under the terms of that
286 version. You may also choose to use such Covered Code under the terms
287 of any subsequent version of the License published by Netscape. No one
288 other than Netscape has the right to modify the terms applicable to
289 Covered Code created under this License.
290
291 6.3. Derivative Works.
292 If You create or use a modified version of this License (which you may
293 only do in order to apply it to code which is not already Covered Code
294 governed by this License), You must (a) rename Your license so that
295 the phrases "Mozilla", "MOZILLAPL", "MOZPL", "Netscape",
296 "MPL", "NPL" or any confusingly similar phrase do not appear in your
297 license (except to note that your license differs from this License)
298 and (b) otherwise make it clear that Your version of the license
299 contains terms which differ from the Mozilla Public License and
300 Netscape Public License. (Filling in the name of the Initial
301 Developer, Original Code or Contributor in the notice described in
302 Exhibit A shall not of themselves be deemed to be modifications of
303 this License.)
304
305 7. DISCLAIMER OF WARRANTY.
306
307 COVERED CODE IS PROVIDED UNDER THIS LICENSE ON AN "AS IS" BASIS,
308 WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING,
309 WITHOUT LIMITATION, WARRANTIES THAT THE COVERED CODE IS FREE OF
310 DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR PURPOSE OR NON-INFRINGING.
311 THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE COVERED CODE
312 IS WITH YOU. SHOULD ANY COVERED CODE PROVE DEFECTIVE IN ANY RESPECT,
313 YOU (NOT THE INITIAL DEVELOPER OR ANY OTHER CONTRIBUTOR) ASSUME THE
314 COST OF ANY NECESSARY SERVICING, REPAIR OR CORRECTION. THIS DISCLAIMER
315 OF WARRANTY CONSTITUTES AN ESSENTIAL PART OF THIS LICENSE. NO USE OF
316 ANY COVERED CODE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS DISCLAIMER.
317
318 8. TERMINATION.
319
320 8.1. This License and the rights granted hereunder will terminate
321 automatically if You fail to comply with terms herein and fail to cure
322 such breach within 30 days of becoming aware of the breach. All
323 sublicenses to the Covered Code which are properly granted shall
324 survive any termination of this License. Provisions which, by their
325 nature, must remain in effect beyond the termination of this License
326 shall survive.
327
328 8.2. If You initiate litigation by asserting a patent infringement
329 claim (excluding declatory judgment actions) against Initial Developer
330 or a Contributor (the Initial Developer or Contributor against whom
331 You file such action is referred to as "Participant") alleging that:
332
333 (a) such Participant's Contributor Version directly or indirectly
334 infringes any patent, then any and all rights granted by such
335 Participant to You under Sections 2.1 and/or 2.2 of this License
336 shall, upon 60 days notice from Participant terminate prospectively,
337 unless if within 60 days after receipt of notice You either: (i)
338 agree in writing to pay Participant a mutually agreeable reasonable
339 royalty for Your past and future use of Modifications made by such
340 Participant, or (ii) withdraw Your litigation claim with respect to
341 the Contributor Version against such Participant. If within 60 days
342 of notice, a reasonable royalty and payment arrangement are not
343 mutually agreed upon in writing by the parties or the litigation claim
344 is not withdrawn, the rights granted by Participant to You under
345 Sections 2.1 and/or 2.2 automatically terminate at the expiration of
346 the 60 day notice period specified above.
347
348 (b) any software, hardware, or device, other than such Participant's
349 Contributor Version, directly or indirectly infringes any patent, then
350 any rights granted to You by such Participant under Sections 2.1(b)
351 and 2.2(b) are revoked effective as of the date You first made, used,
352 sold, distributed, or had made, Modifications made by that
353 Participant.
354
355 8.3. If You assert a patent infringement claim against Participant
356 alleging that such Participant's Contributor Version directly or
357 indirectly infringes any patent where such claim is resolved (such as
358 by license or settlement) prior to the initiation of patent
359 infringement litigation, then the reasonable value of the licenses
360 granted by such Participant under Sections 2.1 or 2.2 shall be taken
361 into account in determining the amount or value of any payment or
362 license.
363
364 8.4. In the event of termination under Sections 8.1 or 8.2 above,
365 all end user license agreements (excluding distributors and resellers)
366 which have been validly granted by You or any distributor hereunder
367 prior to termination shall survive termination.
368
369 9. LIMITATION OF LIABILITY.
370
371 UNDER NO CIRCUMSTANCES AND UNDER NO LEGAL THEORY, WHETHER TORT
372 (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE, SHALL YOU, THE INITIAL
373 DEVELOPER, ANY OTHER CONTRIBUTOR, OR ANY DISTRIBUTOR OF COVERED CODE,
374 OR ANY SUPPLIER OF ANY OF SUCH PARTIES, BE LIABLE TO ANY PERSON FOR
375 ANY INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES OF ANY
376 CHARACTER INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF GOODWILL,
377 WORK STOPPAGE, COMPUTER FAILURE OR MALFUNCTION, OR ANY AND ALL OTHER
378 COMMERCIAL DAMAGES OR LOSSES, EVEN IF SUCH PARTY SHALL HAVE BEEN
379 INFORMED OF THE POSSIBILITY OF SUCH DAMAGES. THIS LIMITATION OF
380 LIABILITY SHALL NOT APPLY TO LIABILITY FOR DEATH OR PERSONAL INJURY
381 RESULTING FROM SUCH PARTY'S NEGLIGENCE TO THE EXTENT APPLICABLE LAW
382 PROHIBITS SUCH LIMITATION. SOME JURISDICTIONS DO NOT ALLOW THE
383 EXCLUSION OR LIMITATION OF INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO
384 THIS EXCLUSION AND LIMITATION MAY NOT APPLY TO YOU.
385
386 10. U.S. GOVERNMENT END USERS.
387
388 The Covered Code is a "commercial item," as that term is defined in
389 48 C.F.R. 2.101 (Oct. 1995), consisting of "commercial computer
390 software" and "commercial computer software documentation," as such
391 terms are used in 48 C.F.R. 12.212 (Sept. 1995). Consistent with 48
392 C.F.R. 12.212 and 48 C.F.R. 227.7202-1 through 227.7202-4 (June 1995),
393 all U.S. Government End Users acquire Covered Code with only those
394 rights set forth herein.
395
396 11. MISCELLANEOUS.
397
398 This License represents the complete agreement concerning subject
399 matter hereof. If any provision of this License is held to be
400 unenforceable, such provision shall be reformed only to the extent
401 necessary to make it enforceable. This License shall be governed by
402 California law provisions (except to the extent applicable law, if
403 any, provides otherwise), excluding its conflict-of-law provisions.
404 With respect to disputes in which at least one party is a citizen of,
405 or an entity chartered or registered to do business in the United
406 States of America, any litigation relating to this License shall be
407 subject to the jurisdiction of the Federal Courts of the Northern
408 District of California, with venue lying in Santa Clara County,
409 California, with the losing party responsible for costs, including
410 without limitation, court costs and reasonable attorneys' fees and
411 expenses. The application of the United Nations Convention on
412 Contracts for the International Sale of Goods is expressly excluded.
413 Any law or regulation which provides that the language of a contract
414 shall be construed against the drafter shall not apply to this
415 License.
416
417 12. RESPONSIBILITY FOR CLAIMS.
418
419 As between Initial Developer and the Contributors, each party is
420 responsible for claims and damages arising, directly or indirectly,
421 out of its utilization of rights under this License and You agree to
422 work with Initial Developer and Contributors to distribute such
423 responsibility on an equitable basis. Nothing herein is intended or
424 shall be deemed to constitute any admission of liability.
425
426 13. MULTIPLE-LICENSED CODE.
427
428 Initial Developer may designate portions of the Covered Code as
429 "Multiple-Licensed". "Multiple-Licensed" means that the Initial
430 Developer permits you to utilize portions of the Covered Code under
431 Your choice of the NPL or the alternative licenses, if any, specified
432 by the Initial Developer in the file described in Exhibit A.
433
434 EXHIBIT A -Mozilla Public License.
435
436 ``The contents of this file are subject to the Mozilla Public License
437 Version 1.1 (the "License"); you may not use this file except in
438 compliance with the License. You may obtain a copy of the License at
439 http://www.mozilla.org/MPL/
440
441 Software distributed under the License is distributed on an "AS IS"
442 basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
443 License for the specific language governing rights and limitations
444 under the License.
445
446 The Original Code is ______________________________________.
447
448 The Initial Developer of the Original Code is ________________________.
449 Portions created by ______________________ are Copyright (C) ______
450 _______________________. All Rights Reserved.
451
452 Contributor(s): ______________________________________.
453
454 Alternatively, the contents of this file may be used under the terms
455 of the _____ license (the "[___] License"), in which case the
456 provisions of [______] License are applicable instead of those
457 above. If you wish to allow use of your version of this file only
458 under the terms of the [____] License and not to allow others to use
459 your version of this file under the MPL, indicate your decision by
460 deleting the provisions above and replace them with the notice and
461 other provisions required by the [___] License. If you do not delete
462 the provisions above, a recipient may use your version of this file
463 under either the MPL or the [___] License."
464
465 [NOTE: The text of this Exhibit A may differ slightly from the text of
466 the notices in the Source Code files of the Original Code. You should
467 use the text of this Exhibit A rather than the text found in the
468 Original Code Source Code for Your Modifications.]
469
+233
-0
README.md less more
0 # juniversalchardet
1
2
3 ## What is it?
4
5 juniversalchardet is a Java port of "universalchardet",
6 that is the encoding detector library of Mozilla.
7
8 Mozilla repo is available at
9 http://hg.mozilla.org/mozilla-central/file/tip/extensions/universalchardet/
10
11
12
13 ## Encodings that can be detected
14
15 - Chinese
16 - ISO-2022-CN
17 - BIG-5
18 - EUC-TW
19 - HZ-GB-2312
20 - GB-18030
21
22 - Cyrillic
23 - ISO-8859-5
24 - KOI8-R
25 - WINDOWS-1251
26 - MACCYRILLIC
27 - IBM866
28 - IBM855
29
30 - Greek
31 - ISO-8859-7
32 - WINDOWS-1253
33
34 - Hebrew
35 - ISO-8859-8
36 - WINDOWS-1255
37
38 - Japanese
39 - ISO-2022-JP
40 - Shift_JIS
41 - EUC-JP
42
43 - Korean
44 - ISO-2022-KR
45 - EUC-KR
46
47 - Unicode
48 - UTF-8
49 - UTF-16BE / UTF-16LE
50 - UTF-32BE / UTF-32LE / X-ISO-10646-UCS-4-3412 / X-ISO-10646-UCS-4-2143
51
52 - Others
53 - WINDOWS-1252
54 - US-ASCII
55
56 All supported encodings are listed in ``org.mozilla.universalchardet.Constants.``
57
58
59 ## How to use it
60
61 (1) Construct an instance of ``org.mozilla.universalchardet.UniversalDetector``.
62
63 (2) Feed some data (typically some thousand bytes) to the detector
64 using ``UniversalDetector.handleData()``.
65
66 (3) Notify the detector of the end of data by using
67 ``UniversalDetector.dataEnd()``.
68
69 (4) Get the detected encoding name by using
70 ``UniversalDetector.getDetectedCharset()``.
71
72 (5) Don't forget to call ``UniversalDetector.reset()`` before you reuse
73 the detector instance for another guess.
74
75
76 ------------ Sample Code ------------
77
78 ```java
79 import org.mozilla.universalchardet.UniversalDetector;
80
81 public class TestDetector
82 {
83 public static void main(String[] args)
84 {
85 byte[] buf = new byte[4096];
86 java.io.InputStream fis = java.nio.file.Files.newInputStream(java.nio.file.Paths.get("test.txt"));
87
88 // (1)
89 UniversalDetector detector = new UniversalDetector();
90
91 // (2)
92 int nread;
93 while ((nread = fis.read(buf)) > 0 && !detector.isDone()) {
94 detector.handleData(buf, 0, nread);
95 }
96 // (3)
97 detector.dataEnd();
98
99 // (4)
100 String encoding = detector.getDetectedCharset();
101 if (encoding != null) {
102 System.out.println("Detected encoding = " + encoding);
103 } else {
104 System.out.println("No encoding detected.");
105 }
106
107 // (5)
108 detector.reset();
109 }
110 }
111 ```
112
113 ### Detecting encoding of a File (simple way)
114
115 ```java
116
117 import org.mozilla.universalchardet.UniversalDetector;
118
119 public class TestDetectorFile {
120
121 public static void main (String[] args) throws java.io.IOException {
122 if (args.length != 1) {
123 System.err.println("Usage: java TestDetectorFile FILENAME");
124 System.exit(1);
125 }
126 java.io.File file = new java.io.File(args[0]);
127 String encoding = UniversalDetector.detectCharset(file);
128 if (encoding != null) {
129 System.out.println("Detected encoding = " + encoding);
130 } else {
131 System.out.println("No encoding detected.");
132 }
133 }
134 }
135
136 ```
137
138 ### Creating a reader with correct encoding
139
140
141 ```java
142
143 import org.mozilla.universalchardet.ReaderFactory;
144
145 public class TestCreateReaderFromFile {
146
147 public static void main (String[] args) throws java.io.IOException {
148 if (args.length != 1) {
149 System.err.println("Usage: java TestCreateReaderFromFile FILENAME");
150 System.exit(1);
151 }
152
153 java.io.Reader reader = null;
154 try {
155 java.io.File file = new java.io.File(args[0]);
156 reader = ReaderFactory.createBufferedReader(file);
157
158 // Do whatever you want with the reader
159 }
160 finally {
161 if (reader != null) {
162 reader.close();
163 }
164 }
165
166 }
167
168 }
169
170 ```
171
172
173 ## Getting with maven
174
175 Put this dependency in your pom.xml
176
177 ```xml
178 <dependency>
179 <groupId>com.github.albfernandez</groupId>
180 <artifactId>juniversalchardet</artifactId>
181 <version>2.4.0</version>
182 </dependency>
183
184 ```
185
186 ## Getting with gradle
187
188 Put this line in your build.gradle
189
190 ```groovy
191 implementation 'com.github.albfernandez:juniversalchardet:2.4.0'
192 ```
193
194 ## Building from sources
195
196 ```bash
197 git clone https://github.com/albfernandez/juniversalchardet.git
198 cd juniversalchardet
199 mvn clean package
200 ```
201
202 ## Related Works
203
204 - jchardet http://jchardet.sourceforge.net/
205
206 jchardet is another Java port of the Mozilla's encoding dectection library.
207 The main difference between jchardet and juniversalchardet is modules
208 they are based on. jchardet is based on the "chardet" module that has
209 long existed. juniversalchardet is based on the "universalchardet" module
210 that is new and generally provides better accuracy on detection results.
211
212 - juniversalchardet https://code.google.com/archive/p/juniversalchardet/
213
214 The original repository of this project
215
216 ## License
217
218 The library is subject to the Mozilla Public License Version 1.1.
219
220 Alternatively, the library may be used under the terms of either
221 the GNU General Public License Version 2 or later, or the GNU
222 Lesser General Public License 2.1 or later.
223
224 ## Compatibility
225
226
227 juniversalchardet requires JDK 7 or higher.
228
229 For Android:
230
231 - juniversalchardet 2.0.x requieres Android 4 (API Level 14)
232 - juniversalchardet 2.1.x and newer requires Android 8 (API Level 26)
+0
-103
build.xml less more
0 <?xml version="1.0" encoding="UTF-8"?>
1 <!--
2 /* ***** BEGIN LICENSE BLOCK *****
3 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
4 *
5 * The contents of this file are subject to the Mozilla Public License Version
6 * 1.1 (the "License"); you may not use this file except in compliance with
7 * the License. You may obtain a copy of the License at
8 * http://www.mozilla.org/MPL/
9 *
10 * Software distributed under the License is distributed on an "AS IS" basis,
11 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
12 * for the specific language governing rights and limitations under the
13 * License.
14 *
15 * The Original Code is mozilla.org code.
16 *
17 * The Initial Developer of the Original Code is
18 * Netscape Communications Corporation.
19 * Portions created by the Initial Developer are Copyright (C) 1998
20 * the Initial Developer. All Rights Reserved.
21 *
22 * Contributor(s):
23 * Kohei TAKETA <k-tak@void.in>
24 * lnezda
25 *
26 * Alternatively, the contents of this file may be used under the terms of
27 * either the GNU General Public License Version 2 or later (the "GPL"), or
28 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
29 * in which case the provisions of the GPL or the LGPL are applicable instead
30 * of those above. If you wish to allow use of your version of this file only
31 * under the terms of either the GPL or the LGPL, and not to allow others to
32 * use your version of this file under the terms of the MPL, indicate your
33 * decision by deleting the provisions above and replace them with the notice
34 * and other provisions required by the GPL or the LGPL. If you do not delete
35 * the provisions above, a recipient may use your version of this file under
36 * the terms of any one of the MPL, the GPL or the LGPL.
37 *
38 * ***** END LICENSE BLOCK ***** */
39 -->
40 <project name="juniversalchardet" default="dist" basedir=".">
41 <description>
42 A Java port of Mozilla's universalchardet.
43 </description>
44
45 <property name="src" location="src"/>
46 <property name="example" location="example"/>
47 <property name="dist" location="dist"/>
48 <property name="version" value="1.0.3"/>
49
50 <path id="example.classpath">
51 <fileset dir="${dist}">
52 <include name="*.jar" />
53 </fileset>
54 <pathelement path="${java.class.path}" />
55 </path>
56
57 <target name="init">
58 <mkdir dir="${dist}" />
59 <mkdir dir="${dist}/classes"/>
60 </target>
61
62 <target name="compile" depends="init">
63 <javac sourcepath=""
64 srcdir="${src}"
65 destdir="${dist}/classes"
66 source="1.5"
67 target="1.5">
68 <include name="**/*.java"/>
69 <exclude name="**/.svn/"/>
70 <exclude name="**/*Test.java"/>
71 </javac>
72 </target>
73
74 <target name="example" depends="dist">
75 <javac sourcepath=""
76 srcdir="${example}"
77 destdir="${example}"
78 classpathref="example.classpath"
79 source="1.5"
80 target="1.5">
81 <include name="**/*.java"/>
82 <exclude name="**/.svn/"/>
83 <exclude name="**/*Test.java"/>
84 </javac>
85 </target>
86
87 <target name="dist" depends="compile">
88 <jar destfile="${dist}/juniversalchardet-${version}.jar"
89 basedir="${dist}/classes"
90 excludes="**/.svn/"
91 />
92 </target>
93
94 <target name="clean">
95 <delete dir="${dist}"/>
96 <delete>
97 <fileset dir="${example}">
98 <include name="*.class" />
99 </fileset>
100 </delete>
101 </target>
102 </project>
+0
-39
data/euc.txt less more
0 ========================================================================
1 コンソール アプリケーション : universalchardet プロジェクトの概要
2 ========================================================================
3
4 この universalchardet アプリケーションは、AppWizard によって作成されました。
5
6 このファイルには、universalchardet アプリケーションを構成する各ファイルの
7 内容の概略が記述されています。
8
9
10 universalchardet.vcproj
11 これは、アプリケーション ウィザードで生成される VC++ プロジェクトのメインの
12 プロジェクト ファイルです。
13 ファイルを生成した Visual C++ のバージョン情報と、アプリケーション
14 ウィザードで選択したプラットフォーム、構成、およびプロジェクトの機能に関する
15 情報が記述されています。
16
17 universalchardet.cpp
18 これは、メインのアプリケーション ソース ファイルです。
19
20 /////////////////////////////////////////////////////////////////////////////
21 その他の標準ファイル :
22
23 StdAfx.h, StdAfx.cpp
24 これらのファイルは、コンパイル済みヘッダー (PCH) ファイル
25 universalchardet.pch とプリコンパイル済み型ファイル StdAfx.obj を
26 ビルドするために使用します。
27
28 /////////////////////////////////////////////////////////////////////////////
29 その他のメモ :
30
31 AppWizard では "TODO:" コメントを使用して、ユーザーが追加またはカスタマイズする
32 ソース部分を示します。
33
34 /////////////////////////////////////////////////////////////////////////////
35
36
37
38
+0
-39
data/iso2022jp.txt less more
0 ========================================================================
1 $B%3%s%=!<%k(J $B%"%W%j%1!<%7%g%s(J : universalchardet $B%W%m%8%'%/%H$N35MW(J
2 ========================================================================
3
4 $B$3$N(J universalchardet $B%"%W%j%1!<%7%g%s$O!"(JAppWizard $B$K$h$C$F:n@.$5$l$^$7$?!#(J
5
6 $B$3$N%U%!%$%k$K$O!"(Juniversalchardet $B%"%W%j%1!<%7%g%s$r9=@.$9$k3F%U%!%$%k$N(J
7 $BFbMF$N35N,$,5-=R$5$l$F$$$^$9!#(J
8
9
10 universalchardet.vcproj
11 $B$3$l$O!"%"%W%j%1!<%7%g%s(J $B%&%#%6!<%I$G@8@.$5$l$k(J VC++ $B%W%m%8%'%/%H$N%a%$%s$N(J
12 $B%W%m%8%'%/%H(J $B%U%!%$%k$G$9!#(J
13 $B%U%!%$%k$r@8@.$7$?(J Visual C++ $B$N%P!<%8%g%s>pJs$H!"%"%W%j%1!<%7%g%s(J
14 $B%&%#%6!<%I$GA*Br$7$?%W%i%C%H%U%)!<%`!"9=@.!"$*$h$S%W%m%8%'%/%H$N5!G=$K4X$9$k(J
15 $B>pJs$,5-=R$5$l$F$$$^$9!#(J
16
17 universalchardet.cpp
18 $B$3$l$O!"%a%$%s$N%"%W%j%1!<%7%g%s(J $B%=!<%9(J $B%U%!%$%k$G$9!#(J
19
20 /////////////////////////////////////////////////////////////////////////////
21 $B$=$NB>$NI8=`%U%!%$%k(J :
22
23 StdAfx.h, StdAfx.cpp
24 $B$3$l$i$N%U%!%$%k$O!"%3%s%Q%$%k:Q$_%X%C%@!<(J (PCH) $B%U%!%$%k(J
25 universalchardet.pch $B$H%W%j%3%s%Q%$%k:Q$_7?%U%!%$%k(J StdAfx.obj $B$r(J
26 $B%S%k%I$9$k$?$a$K;HMQ$7$^$9!#(J
27
28 /////////////////////////////////////////////////////////////////////////////
29 $B$=$NB>$N%a%b(J :
30
31 AppWizard $B$G$O(J "TODO:" $B%3%a%s%H$r;HMQ$7$F!"%f!<%6!<$,DI2C$^$?$O%+%9%?%^%$%:$9$k(J
32 $B%=!<%9ItJ,$r<($7$^$9!#(J
33
34 /////////////////////////////////////////////////////////////////////////////
35
36
37
38
+0
-39
data/shiftjis.txt less more
0 ========================================================================
1 コンソール アプリケーション : universalchardet プロジェクトの概要
2 ========================================================================
3
4 この universalchardet アプリケーションは、AppWizard によって作成されました。
5
6 このファイルには、universalchardet アプリケーションを構成する各ファイルの
7 内容の概略が記述されています。
8
9
10 universalchardet.vcproj
11 これは、アプリケーション ウィザードで生成される VC++ プロジェクトのメインの
12 プロジェクト ファイルです。
13 ファイルを生成した Visual C++ のバージョン情報と、アプリケーション
14 ウィザードで選択したプラットフォーム、構成、およびプロジェクトの機能に関する
15 î•ñ‚ª‹Lq‚³‚ê‚Ä‚¢‚Ü‚·B
16
17 universalchardet.cpp
18 これは、メインのアプリケーション ソース ファイルです。
19
20 /////////////////////////////////////////////////////////////////////////////
21 その他の標準ファイル :
22
23 StdAfx.h, StdAfx.cpp
24 これらのファイルは、コンパイル済みヘッダー (PCH) ファイル
25 universalchardet.pch とプリコンパイル済み型ファイル StdAfx.obj を
26 ビルドするために使用します。
27
28 /////////////////////////////////////////////////////////////////////////////
29 ‚»‚Ì‘¼‚̃ƒ‚ :
30
31 AppWizard では "TODO:" コメントを使用して、ユーザーが追加またはカスタマイズする
32 ソース部分を示します。
33
34 /////////////////////////////////////////////////////////////////////////////
35
36
37
38
data/utf16le.txt less more
Binary diff not shown
+0
-39
data/utf8.txt less more
0 ========================================================================
1 コンソール アプリケーション : universalchardet プロジェクトの概要
2 ========================================================================
3
4 この universalchardet アプリケーションは、AppWizard によって作成されました。
5
6 このファイルには、universalchardet アプリケーションを構成する各ファイルの
7 内容の概略が記述されています。
8
9
10 universalchardet.vcproj
11 これは、アプリケーション ウィザードで生成される VC++ プロジェクトのメインの
12 プロジェクト ファイルです。
13 ファイルを生成した Visual C++ のバージョン情報と、アプリケーション
14 ウィザードで選択したプラットフォーム、構成、およびプロジェクトの機能に関する
15 情報が記述されています。
16
17 universalchardet.cpp
18 これは、メインのアプリケーション ソース ファイルです。
19
20 /////////////////////////////////////////////////////////////////////////////
21 その他の標準ファイル :
22
23 StdAfx.h, StdAfx.cpp
24 これらのファイルは、コンパイル済みヘッダー (PCH) ファイル
25 universalchardet.pch とプリコンパイル済み型ファイル StdAfx.obj を
26 ビルドするために使用します。
27
28 /////////////////////////////////////////////////////////////////////////////
29 その他のメモ :
30
31 AppWizard では "TODO:" コメントを使用して、ユーザーが追加またはカスタマイズする
32 ソース部分を示します。
33
34 /////////////////////////////////////////////////////////////////////////////
35
36
37
38
+0
-39
data/utf8n.txt less more
0 ========================================================================
1 コンソール アプリケーション : universalchardet プロジェクトの概要
2 ========================================================================
3
4 この universalchardet アプリケーションは、AppWizard によって作成されました。
5
6 このファイルには、universalchardet アプリケーションを構成する各ファイルの
7 内容の概略が記述されています。
8
9
10 universalchardet.vcproj
11 これは、アプリケーション ウィザードで生成される VC++ プロジェクトのメインの
12 プロジェクト ファイルです。
13 ファイルを生成した Visual C++ のバージョン情報と、アプリケーション
14 ウィザードで選択したプラットフォーム、構成、およびプロジェクトの機能に関する
15 情報が記述されています。
16
17 universalchardet.cpp
18 これは、メインのアプリケーション ソース ファイルです。
19
20 /////////////////////////////////////////////////////////////////////////////
21 その他の標準ファイル :
22
23 StdAfx.h, StdAfx.cpp
24 これらのファイルは、コンパイル済みヘッダー (PCH) ファイル
25 universalchardet.pch とプリコンパイル済み型ファイル StdAfx.obj を
26 ビルドするために使用します。
27
28 /////////////////////////////////////////////////////////////////////////////
29 その他のメモ :
30
31 AppWizard では "TODO:" コメントを使用して、ユーザーが追加またはカスタマイズする
32 ソース部分を示します。
33
34 /////////////////////////////////////////////////////////////////////////////
35
36
37
38
+0
-75
example/TestDetector.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in>
22 * lnezda
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37 import org.mozilla.universalchardet.UniversalDetector;
38
39 public class TestDetector
40 {
41 public static void main(String[] args) throws java.io.IOException
42 {
43 if (args.length != 1) {
44 System.err.println("Usage: java TestDetector FILENAME");
45 System.exit(1);
46 }
47
48 byte[] buf = new byte[4096];
49 String fileName = args[0];
50 java.io.FileInputStream fis = new java.io.FileInputStream(fileName);
51
52 // (1)
53 UniversalDetector detector = new UniversalDetector(null);
54
55 // (2)
56 int nread;
57 while ((nread = fis.read(buf)) > 0 && !detector.isDone()) {
58 detector.handleData(buf, 0, nread);
59 }
60 // (3)
61 detector.dataEnd();
62
63 // (4)
64 String encoding = detector.getDetectedCharset();
65 if (encoding != null) {
66 System.out.println("Detected encoding = " + encoding);
67 } else {
68 System.out.println("No encoding detected.");
69 }
70
71 // (5)
72 detector.reset();
73 }
74 }
+11
-0
mozilla_repositories.txt less more
0 Original mozilla repo:
1
2 http://lxr.mozilla.org/seamonkey/source/extensions/universalchardet/
3
4 ¿new mozilla repo?
5 http://hg.mozilla.org/mozilla-central/file/tip/extensions/universalchardet/
6
7
8 Multiple charsets removed
9 cause: Remove encoding detectors that are no longer offered in the UI. r=emk.
10 http://hg.mozilla.org/mozilla-central/rev/2f0c68d4d7ca⏎
+264
-0
pom.xml less more
0 <project xmlns="http://maven.apache.org/POM/4.0.0"
1 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
2 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
3 <modelVersion>4.0.0</modelVersion>
4 <groupId>com.github.albfernandez</groupId>
5 <artifactId>juniversalchardet</artifactId>
6 <version>2.4.0</version>
7 <url>https://github.com/albfernandez/juniversalchardet</url>
8 <packaging>jar</packaging>
9 <name>juniversalchardet</name>
10 <description>JUniversalChardet is a Java encoding detector library</description>
11
12 <licenses>
13 <license>
14 <name>Mozilla Public License Version 1.1</name>
15 <url>https://www.mozilla.org/en-US/MPL/1.1/</url>
16 </license>
17 <license>
18 <name>GENERAL PUBLIC LICENSE, version 3 (GPL-3.0)</name>
19 <url>http://www.gnu.org/licenses/gpl.txt</url>
20 </license>
21 <license>
22 <name>GNU LESSER GENERAL PUBLIC LICENSE, version 3 (LGPL-3.0)</name>
23 <url>http://www.gnu.org/licenses/lgpl.txt</url>
24 </license>
25 </licenses>
26 <developers>
27 <developer>
28 <name>Alberto Fernández</name>
29 <email>infjaf@gmail.com</email>
30 <organization>Alberto Fernández</organization>
31 <organizationUrl>https://github.com/albfernandez/</organizationUrl>
32 </developer>
33 </developers>
34 <scm>
35 <connection>scm:git:git@github.com:albfernandez/juniversalchardet.git</connection>
36 <developerConnection>scm:git:git@github.com:albfernandez/juniversalchardet.git</developerConnection>
37 <url>git@github.com:albfernandez/juniversalchardet.git</url>
38 </scm>
39 <issueManagement>
40 <system>GitHub</system>
41 <url>https://github.com/albfernandez/juniversalchardet/issues</url>
42 </issueManagement>
43
44 <properties>
45 <compiler.level>1.7</compiler.level>
46 <compiler.module.level>9</compiler.module.level>
47 <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
48 <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
49
50 <version.spotbugs-maven-plugin>4.1.4</version.spotbugs-maven-plugin>
51 <version.findsecbugs-plugin>1.11.0</version.findsecbugs-plugin>
52 <version.sb-contrib>7.4.7</version.sb-contrib>
53 </properties>
54
55
56 <dependencies>
57 <dependency>
58 <groupId>junit</groupId>
59 <artifactId>junit</artifactId>
60 <version>4.13.1</version>
61 <scope>test</scope>
62 </dependency>
63 <dependency>
64 <groupId>commons-codec</groupId>
65 <artifactId>commons-codec</artifactId>
66 <version>1.10</version>
67 <scope>test</scope>
68 </dependency>
69 </dependencies>
70
71
72 <build>
73 <plugins>
74
75 <plugin>
76 <groupId>org.apache.maven.plugins</groupId>
77 <artifactId>maven-jar-plugin</artifactId>
78 <version>3.0.2</version>
79 <configuration>
80 <archive>
81 <manifestEntries>
82 <Automatic-Module-Name>com.github.albfernandez.juniversalchardet</Automatic-Module-Name>
83 </manifestEntries>
84 <manifest>
85 <addDefaultImplementationEntries>true</addDefaultImplementationEntries>
86 <addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
87 </manifest>
88 </archive>
89 </configuration>
90 </plugin>
91 <plugin>
92 <groupId>org.apache.maven.plugins</groupId>
93 <artifactId>maven-source-plugin</artifactId>
94 <version>3.0.1</version>
95 <executions>
96 <execution>
97 <id>attach-sources</id>
98 <goals>
99 <goal>jar</goal>
100 </goals>
101 </execution>
102 </executions>
103 </plugin>
104 <plugin>
105 <groupId>org.apache.maven.plugins</groupId>
106 <artifactId>maven-javadoc-plugin</artifactId>
107 <version>3.2.0</version>
108 <executions>
109 <execution>
110 <id>attach-javadocs</id>
111 <goals>
112 <goal>jar</goal>
113 </goals>
114 <configuration>
115 <source>${compiler.level}</source>
116 </configuration>
117 </execution>
118 </executions>
119 </plugin>
120 <plugin>
121 <groupId>org.apache.maven.plugins</groupId>
122 <artifactId>maven-gpg-plugin</artifactId>
123 <version>1.6</version>
124 <configuration>
125 <keyname>${gpg.keyname}</keyname>
126 <passphraseServerId>${gpg.keyname}</passphraseServerId>
127 </configuration>
128 <executions>
129 <execution>
130 <id>sign-artifacts</id>
131 <phase>verify</phase>
132 <goals>
133 <goal>sign</goal>
134 </goals>
135 </execution>
136 </executions>
137 </plugin>
138 <plugin>
139 <groupId>org.sonatype.plugins</groupId>
140 <artifactId>nexus-staging-maven-plugin</artifactId>
141 <version>1.6.6</version>
142 <extensions>true</extensions>
143 <configuration>
144 <serverId>ossrh</serverId>
145 <nexusUrl>https://oss.sonatype.org/</nexusUrl>
146 <autoReleaseAfterClose>true</autoReleaseAfterClose>
147 </configuration>
148 </plugin>
149 </plugins>
150 </build>
151
152
153 <profiles>
154 <profile>
155 <id>jdk7on</id>
156 <activation>
157 <jdk>(,1.8]</jdk>
158 </activation>
159 <build>
160 <plugins>
161 <plugin>
162 <groupId>org.apache.maven.plugins</groupId>
163 <artifactId>maven-compiler-plugin</artifactId>
164 <version>3.7.0</version>
165 <configuration>
166 <excludes>
167 <exclude>module-info.java</exclude>
168 </excludes>
169 <source>${compiler.level}</source>
170 <target>${compiler.level}</target>
171 </configuration>
172 </plugin>
173 <plugin>
174 <groupId>org.codehaus.mojo</groupId>
175 <artifactId>findbugs-maven-plugin</artifactId>
176 <version>3.0.5</version>
177 <executions>
178 <execution>
179 <id>findbugs-check</id>
180 <phase>verify</phase>
181 <goals>
182 <goal>check</goal>
183 </goals>
184 </execution>
185 </executions>
186 </plugin>
187 </plugins>
188 </build>
189 </profile>
190 <profile>
191 <id>jdk9on</id>
192 <activation>
193 <jdk>[9,)</jdk>
194 </activation>
195 <build>
196 <plugins>
197 <plugin>
198 <groupId>org.apache.maven.plugins</groupId>
199 <artifactId>maven-compiler-plugin</artifactId>
200 <version>3.8.1</version>
201 <executions>
202 <execution>
203 <id>default-compile</id>
204 <configuration>
205 <release>${compiler.module.level}</release>
206 </configuration>
207 </execution>
208 <execution>
209 <id>base-compile</id>
210 <goals>
211 <goal>compile</goal>
212 </goals>
213 <configuration>
214 <excludes>
215 <exclude>module-info.java</exclude>
216 </excludes>
217 </configuration>
218 </execution>
219 </executions>
220 <configuration>
221 <source>${compiler.level}</source>
222 <target>${compiler.level}</target>
223 </configuration>
224 </plugin>
225 <plugin>
226 <groupId>com.github.spotbugs</groupId>
227 <artifactId>spotbugs-maven-plugin</artifactId>
228 <version>${version.spotbugs-maven-plugin}</version>
229 <configuration>
230 <effort>Max</effort>
231 <threshold>Medium</threshold>
232 <includeTests>false</includeTests>
233 <failOnError>false</failOnError>
234 <plugins>
235 <plugin>
236 <groupId>com.h3xstream.findsecbugs</groupId>
237 <artifactId>findsecbugs-plugin</artifactId>
238 <version>${version.findsecbugs-plugin}</version>
239 </plugin>
240 <plugin>
241 <groupId>com.mebigfatguy.sb-contrib</groupId>
242 <artifactId>sb-contrib</artifactId>
243 <version>${version.sb-contrib}</version>
244 </plugin>
245 </plugins>
246 </configuration>
247 <executions>
248 <execution>
249 <id>spotbugs-check</id>
250 <phase>verify</phase>
251 <goals>
252 <goal>check</goal>
253 </goals>
254 </execution>
255 </executions>
256 </plugin>
257 </plugins>
258 </build>
259 </profile>
260
261 </profiles>
262
263 </project>
+0
-127
readme.txt less more
0 juniversalchardet
1
2
3 1. What is it?
4
5 juniversalchardet is a Java port of "universalchardet",
6 that is the encoding detector library of Mozilla.
7
8 The original code of universalchardet is available at
9 http://lxr.mozilla.org/seamonkey/source/extensions/universalchardet/
10
11 Techniques used by universalchardet are described at
12 http://www.mozilla.org/projects/intl/UniversalCharsetDetection.html
13
14
15 2. Encodings that can be detected
16
17 - Chinese
18 - ISO-2022-CN
19 - BIG-5
20 - EUC-TW
21 - GB18030
22 - HZ-GB-2312
23
24 - Cyrillic
25 - ISO-8859-5
26 - KOI8-R
27 - WINDOWS-1251
28 - MACCYRILLIC
29 - IBM866
30 - IBM855
31
32 - Greek
33 - ISO-8859-7
34 - WINDOWS-1253
35
36 - Hebrew
37 - ISO-8859-8
38 - WINDOWS-1255
39
40 - Japanese
41 - ISO-2022-JP
42 - Shift_JIS
43 - EUC-JP
44
45 - Korean
46 - ISO-2022-KR
47 - EUC-KR
48
49 - Unicode
50 - UTF-8
51 - UTF-16BE / UTF-16LE
52 - UTF-32BE / UTF-32LE / X-ISO-10646-UCS-4-3412 / X-ISO-10646-UCS-4-2143
53
54 - Others
55 - WINDOWS-1252
56
57 All supported encodings are listed in org.mozilla.universalchardet.Constants.
58
59
60 3. How to use it
61
62 (1) Construct an instance of org.mozilla.universalchardet.UniversalDetector.
63 (2) Feed some data (typically some thousand bytes) to the detector
64 using UniversalDetector.handleData().
65 (3) Notify the detector of the end of data by using
66 UniversalDetector.dataEnd().
67 (4) Get the detected encoding name by using
68 UniversalDetector.getDetectedCharset().
69 (5) Don't forget to call UniversalDetector.reset() before you reuse
70 the detector instance for another guess.
71
72
73 ------------ Sample Code ------------
74 import org.mozilla.universalchardet.UniversalDetector;
75
76 public class TestDetector
77 {
78 public static void main(String[] args)
79 {
80 byte[] buf = new byte[4096];
81 java.io.FileInputStream fis = new java.io.FileInputStream("test.txt");
82
83 // (1)
84 UniversalDetector detector = new UniversalDetector(null);
85
86 // (2)
87 int nread;
88 while ((nread = fis.read(buf)) > 0 && !detector.isDone()) {
89 detector.handleData(buf, 0, nread);
90 }
91 // (3)
92 detector.dataEnd();
93
94 // (4)
95 String encoding = detector.getDetectedCharset();
96 if (encoding != null) {
97 System.out.println("Detected encoding = " + encoding);
98 } else {
99 System.out.println("No encoding detected.");
100 }
101
102 // (5)
103 detector.reset();
104 }
105 }
106
107
108 4. Related Woks
109
110 - jchardet http://jchardet.sourceforge.net/
111
112 jchardet is another Java port of the Mozilla's encoding dectection library.
113 The main difference between jchardet and juniversalchardet is modules
114 they are based on. jchardet is based on the "chardet" module that has
115 long existed. juniversalchardet is based on the "universalchardet" module
116 that is new and generally provides better accuracy on detection results.
117
118
119 5. License
120
121 The library is subject to the Mozilla Public License Version 1.1.
122 Alternatively, the library may be used under the terms of either
123 the GNU General Public License Version 2 or later, or the GNU
124 Lesser General Public License 2.1 or later.
125
126
+8
-0
src/main/java/module-info.java less more
0 module com.github.albfernandez.juniversalchardet {
1 exports org.mozilla.universalchardet;
2 exports org.mozilla.universalchardet.prober;
3 exports org.mozilla.universalchardet.prober.contextanalysis;
4 exports org.mozilla.universalchardet.prober.distributionanalysis;
5 exports org.mozilla.universalchardet.prober.sequence;
6 exports org.mozilla.universalchardet.prober.statemachine;
7 }⏎
+51
-0
src/main/java/org/mozilla/universalchardet/CharsetListener.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet;
38
39 /**
40 * Listener to be notified when Charset is detected.
41 *
42 */
43 public interface CharsetListener {
44
45 /**
46 * Notifies a charset is detected
47 * @param charset The charset being detected
48 */
49 void report(String charset);
50 }
+66
-0
src/main/java/org/mozilla/universalchardet/Constants.java less more
0 /*
1
2 (C) Copyright 2016-2017 Alberto Fernández <infjaf@gmail.com>
3 (C) Copyright 2007 Kohei TAKETA <k-tak@void.in> (Java port)
4
5 The contents of this file are subject to the Mozilla Public License Version
6 1.1 (the "License"); you may not use this file except in compliance with
7 the License. You may obtain a copy of the License at
8 http://www.mozilla.org/MPL/
9
10 Software distributed under the License is distributed on an "AS IS" basis,
11 WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
12 for the specific language governing rights and limitations under the
13 License.
14
15 Alternatively, the contents of this file may be used under the terms of
16 either the GNU General Public License Version 2 or later (the "GPL"), or
17 the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
18 in which case the provisions of the GPL or the LGPL are applicable instead
19 of those above. If you wish to allow use of your version of this file only
20 under the terms of either the GPL or the LGPL, and not to allow others to
21 use your version of this file under the terms of the MPL, indicate your
22 decision by deleting the provisions above and replace them with the notice
23 and other provisions required by the GPL or the LGPL. If you do not delete
24 the provisions above, a recipient may use your version of this file under
25 the terms of any one of the MPL, the GPL or the LGPL.
26
27 */
28
29 package org.mozilla.universalchardet;
30
31 public final class Constants {
32 public static final String CHARSET_ISO_2022_JP = "ISO-2022-JP".intern();
33 public static final String CHARSET_ISO_2022_CN = "ISO-2022-CN".intern();
34 public static final String CHARSET_ISO_2022_KR = "ISO-2022-KR".intern();
35 public static final String CHARSET_ISO_8859_5 = "ISO-8859-5".intern();
36 public static final String CHARSET_ISO_8859_7 = "ISO-8859-7".intern();
37 public static final String CHARSET_ISO_8859_8 = "ISO-8859-8".intern();
38 public static final String CHARSET_BIG5 = "BIG5".intern();
39 public static final String CHARSET_GB18030 = "GB18030".intern();
40 public static final String CHARSET_EUC_JP = "EUC-JP".intern();
41 public static final String CHARSET_EUC_KR = "EUC-KR".intern();
42 public static final String CHARSET_EUC_TW = "EUC-TW".intern();
43 public static final String CHARSET_SHIFT_JIS = "SHIFT_JIS".intern();
44 public static final String CHARSET_IBM855 = "IBM855".intern();
45 public static final String CHARSET_IBM866 = "IBM866".intern();
46 public static final String CHARSET_KOI8_R = "KOI8-R".intern();
47 public static final String CHARSET_MACCYRILLIC = "MACCYRILLIC".intern();
48 public static final String CHARSET_WINDOWS_1251 = "WINDOWS-1251".intern();
49 public static final String CHARSET_WINDOWS_1252 = "WINDOWS-1252".intern();
50 public static final String CHARSET_WINDOWS_1253 = "WINDOWS-1253".intern();
51 public static final String CHARSET_WINDOWS_1255 = "WINDOWS-1255".intern();
52 public static final String CHARSET_UTF_8 = "UTF-8".intern();
53 public static final String CHARSET_UTF_16BE = "UTF-16BE".intern();
54 public static final String CHARSET_UTF_16LE = "UTF-16LE".intern();
55 public static final String CHARSET_UTF_32BE = "UTF-32BE".intern();
56 public static final String CHARSET_UTF_32LE = "UTF-32LE".intern();
57 public static final String CHARSET_TIS620 = "TIS620".intern();
58 public static final String CHARSET_US_ASCCI = "US-ASCII".intern();
59
60
61 // WARNING: Listed below are charsets which Java does not support.
62 public static final String CHARSET_HZ_GB_2312 = "HZ-GB-2312".intern(); // Simplified Chinese
63 public static final String CHARSET_X_ISO_10646_UCS_4_3412 = "X-ISO-10646-UCS-4-3412".intern(); // Malformed UTF-32
64 public static final String CHARSET_X_ISO_10646_UCS_4_2143 = "X-ISO-10646-UCS-4-2143".intern(); // Malformed UTF-32
65 }
+118
-0
src/main/java/org/mozilla/universalchardet/EncodingDetectorInputStream.java less more
0 /*
1
2 (C) Copyright 2016-2017 Alberto Fernández <infjaf@gmail.com>
3
4 The contents of this file are subject to the Mozilla Public License Version
5 1.1 (the "License"); you may not use this file except in compliance with
6 the License. You may obtain a copy of the License at
7 http://www.mozilla.org/MPL/
8
9 Software distributed under the License is distributed on an "AS IS" basis,
10 WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
11 for the specific language governing rights and limitations under the
12 License.
13
14 Alternatively, the contents of this file may be used under the terms of
15 either the GNU General Public License Version 2 or later (the "GPL"), or
16 the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
17 in which case the provisions of the GPL or the LGPL are applicable instead
18 of those above. If you wish to allow use of your version of this file only
19 under the terms of either the GPL or the LGPL, and not to allow others to
20 use your version of this file under the terms of the MPL, indicate your
21 decision by deleting the provisions above and replace them with the notice
22 and other provisions required by the GPL or the LGPL. If you do not delete
23 the provisions above, a recipient may use your version of this file under
24 the terms of any one of the MPL, the GPL or the LGPL.
25
26 */
27 package org.mozilla.universalchardet;
28
29 import java.io.IOException;
30 import java.io.InputStream;
31
32 /**
33 * Stream that detect encoding while reading.
34 * The normal usage is to fully read from inputstream and call close before check for charset.
35 *
36 */
37 public class EncodingDetectorInputStream extends InputStream {
38
39 private InputStream in;
40 private final UniversalDetector detector = new UniversalDetector(null);
41
42 /**
43 * Create the stream
44 * @param in The InputStream to read from
45 */
46 public EncodingDetectorInputStream(InputStream in) {
47 this.in = in;
48 }
49
50 public int available() throws IOException {
51 return in.available();
52 }
53
54 public void close() throws IOException {
55 in.close();
56 }
57
58 public void mark(int readlimit) {
59 in.mark(readlimit);
60 }
61
62 public boolean markSupported() {
63 return in.markSupported();
64 }
65
66 public int read() throws IOException {
67 byte[] data = new byte[1];
68 int nrOfBytesRead = this.read(data, 0, 1);
69 if (nrOfBytesRead >= 0){
70 return data[0];
71 }
72 return -1;
73 }
74
75 public int read(byte[] b, int off, int len) throws IOException {
76 final int nrOfBytesRead = in.read(b, off, len);
77 if (!detector.isDone() && nrOfBytesRead > 0) {
78 detector.handleData(b, off, nrOfBytesRead);
79 }
80 if (nrOfBytesRead == -1) {
81 detector.dataEnd();
82 }
83 return nrOfBytesRead;
84 }
85
86 public int read(byte[] b) throws IOException {
87 return this.read(b, 0, b.length);
88 }
89
90 public void reset() throws IOException {
91 in.reset();
92 }
93
94 public long skip(long n) throws IOException {
95 if (detector.isDone()) {
96 return in.skip(n);
97 }
98 else {
99 int lastRead = 0;
100 long count = -1;
101 for (long i = 0; i < n && lastRead >= 0; i++) {
102 lastRead = this.in.read();
103 count++;
104 }
105 return count;
106 }
107 }
108
109 /**
110 * Gets the detected charset, null if not yet detected.
111 * @return The detected charset
112 */
113 public String getDetectedCharset() {
114 return detector.getDetectedCharset();
115 }
116
117 }
+82
-0
src/main/java/org/mozilla/universalchardet/EncodingDetectorOutputStream.java less more
0 /*
1
2 (C) Copyright 2016-2017 Alberto Fernández <infjaf@gmail.com>
3
4 The contents of this file are subject to the Mozilla Public License Version
5 1.1 (the "License"); you may not use this file except in compliance with
6 the License. You may obtain a copy of the License at
7 http://www.mozilla.org/MPL/
8
9 Software distributed under the License is distributed on an "AS IS" basis,
10 WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
11 for the specific language governing rights and limitations under the
12 License.
13
14 Alternatively, the contents of this file may be used under the terms of
15 either the GNU General Public License Version 2 or later (the "GPL"), or
16 the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
17 in which case the provisions of the GPL or the LGPL are applicable instead
18 of those above. If you wish to allow use of your version of this file only
19 under the terms of either the GPL or the LGPL, and not to allow others to
20 use your version of this file under the terms of the MPL, indicate your
21 decision by deleting the provisions above and replace them with the notice
22 and other provisions required by the GPL or the LGPL. If you do not delete
23 the provisions above, a recipient may use your version of this file under
24 the terms of any one of the MPL, the GPL or the LGPL.
25
26 */
27
28 package org.mozilla.universalchardet;
29
30 import java.io.IOException;
31 import java.io.OutputStream;
32
33 /**
34 * Stream that detect encoding while reading.
35 * The normal usage is to fully read from inputstream and call close before check for charset.
36 *
37 */
38 public class EncodingDetectorOutputStream extends OutputStream {
39 private OutputStream out;
40 private final UniversalDetector detector = new UniversalDetector(null);
41
42 public EncodingDetectorOutputStream(OutputStream out) {
43 super();
44 this.out = out;
45 }
46
47 public void close() throws IOException {
48 out.close();
49 detector.dataEnd();
50 }
51
52
53
54 public void flush() throws IOException {
55 out.flush();
56 }
57
58 public void write(byte[] b, int off, int len) throws IOException {
59 out.write(b, off, len);
60 if (!detector.isDone()) {
61 detector.handleData(b, off, len);
62 }
63
64 }
65
66 public void write(byte[] b) throws IOException {
67 this.write(b,0, b.length);
68 }
69
70 public void write(int b) throws IOException {
71 this.write(new byte[]{(byte) b});
72 }
73 /**
74 * Gets the detected charset, null if not yet detected.
75 * @return The detected charset
76 */
77 public String getDetectedCharset() {
78 return detector.getDetectedCharset();
79 }
80
81 }
+106
-0
src/main/java/org/mozilla/universalchardet/ReaderFactory.java less more
0 /*
1
2 (C) Copyright 2016-2017 Alberto Fernández <infjaf@gmail.com>
3
4 The contents of this file are subject to the Mozilla Public License Version
5 1.1 (the "License"); you may not use this file except in compliance with
6 the License. You may obtain a copy of the License at
7 http://www.mozilla.org/MPL/
8
9 Software distributed under the License is distributed on an "AS IS" basis,
10 WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
11 for the specific language governing rights and limitations under the
12 License.
13
14 Alternatively, the contents of this file may be used under the terms of
15 either the GNU General Public License Version 2 or later (the "GPL"), or
16 the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
17 in which case the provisions of the GPL or the LGPL are applicable instead
18 of those above. If you wish to allow use of your version of this file only
19 under the terms of either the GPL or the LGPL, and not to allow others to
20 use your version of this file under the terms of the MPL, indicate your
21 decision by deleting the provisions above and replace them with the notice
22 and other provisions required by the GPL or the LGPL. If you do not delete
23 the provisions above, a recipient may use your version of this file under
24 the terms of any one of the MPL, the GPL or the LGPL.
25
26 */
27
28 package org.mozilla.universalchardet;
29
30 import java.io.BufferedInputStream;
31 import java.io.BufferedReader;
32 import java.io.File;
33 import java.io.IOException;
34 import java.io.InputStreamReader;
35 import java.io.Reader;
36 import java.nio.charset.Charset;
37 import java.nio.file.Files;
38 import java.nio.file.Path;
39 import java.util.Objects;
40
41 /**
42 * Create a reader from a file with correct encoding
43 */
44 public final class ReaderFactory {
45
46 private ReaderFactory() {
47 throw new AssertionError("No instances allowed");
48 }
49 /**
50 * Create a reader from a file with correct encoding
51 * @param file The file to read from
52 * @param defaultCharset defaultCharset to use if can't be determined
53 * @return BufferedReader for the file with the correct encoding
54 * @throws java.io.IOException if some I/O error ocurrs
55 */
56
57 public static BufferedReader createBufferedReader(File file, Charset defaultCharset) throws IOException {
58 Charset cs = Objects.requireNonNull(defaultCharset, "defaultCharset must be not null");
59 String detectedEncoding = UniversalDetector.detectCharset(file);
60 if (detectedEncoding != null) {
61 cs = Charset.forName(detectedEncoding);
62 }
63 if (!cs.name().contains("UTF")) {
64 return Files.newBufferedReader(file.toPath(), cs);
65 }
66 Path path = file.toPath();
67 return new BufferedReader(new InputStreamReader(new UnicodeBOMInputStream(new BufferedInputStream(Files.newInputStream(path))), cs));
68 }
69 /**
70 * Create a reader from a file with correct encoding. If charset cannot be determined,
71 * it uses the system default charset.
72 * @param file The file to read from
73 * @return BufferedReader for the file with the correct encoding
74 * @throws java.io.IOException if some I/O error ocurrs
75 */
76 public static BufferedReader createBufferedReader(File file) throws IOException {
77 return createBufferedReader(file, Charset.defaultCharset());
78 }
79
80 /**
81 * Create a reader from a file with the correct encoding
82 * @param file The file to read from
83 * @param defaultCharset defaultCharset to use if can't be determined
84 * @return Reader for the file with the correct encoding
85 * @throws java.io.IOException if some I/O error ocurrs
86 * @deprecated Use {@link #createBufferedReader(File, Charset)}
87 *
88 */
89 @Deprecated
90 public static Reader createReaderFromFile(File file, Charset defaultCharset) throws IOException {
91 return createBufferedReader(file, defaultCharset);
92 }
93 /**
94 * Create a reader from a file with the correct encoding. If charset cannot be determined,
95 * it uses the system default charset.
96 * @param file The file to read from
97 * @return Reader for the file with the correct encoding
98 * @throws java.io.IOException if some I/O error ocurrs
99 * @deprecated Use {@link #createBufferedReader(File)}
100 */
101 @Deprecated
102 public static Reader createReaderFromFile(File file) throws IOException {
103 return createReaderFromFile(file, Charset.defaultCharset());
104 }
105 }
+297
-0
src/main/java/org/mozilla/universalchardet/UnicodeBOMInputStream.java less more
0 // (‑●‑●)> released under the WTFPL v2 license, by Gregory Pakosz (@gpakosz)
1
2 package org.mozilla.universalchardet;
3
4 import java.io.IOException;
5 import java.io.InputStream;
6 import java.io.PushbackInputStream;
7
8 /**
9 * The <code>UnicodeBOMInputStream</code> class wraps any
10 * <code>InputStream</code> and detects the presence of any Unicode BOM
11 * (Byte Order Mark) at its beginning, as defined by
12 * <a href="http://www.faqs.org/rfcs/rfc3629.html">RFC 3629 - UTF-8, a
13 * transformation format of ISO 10646</a>
14 *
15 * <p>The
16 * <a href="http://www.unicode.org/unicode/faq/utf_bom.html">Unicode FAQ</a>
17 * defines 5 types of BOMs:</p><ul>
18 * <li><pre>00 00 FE FF = UTF-32, big-endian</pre></li>
19 * <li><pre>FF FE 00 00 = UTF-32, little-endian</pre></li>
20 * <li><pre>FE FF = UTF-16, big-endian</pre></li>
21 * <li><pre>FF FE = UTF-16, little-endian</pre></li>
22 * <li><pre>EF BB BF = UTF-8</pre></li>
23 * </ul>
24 *
25 * <p>Use the {@link #getBOM()} method to know whether a BOM has been detected
26 * or not.
27 * </p>
28 * <p>Use the {@link #skipBOM()} method to remove the detected BOM from the
29 * wrapped <code>InputStream</code> object.</p>
30 *
31 * @author Gregory Pakosz
32 * @version 1.0
33 */
34 public class UnicodeBOMInputStream extends InputStream {
35 /**
36 * Type safe enumeration class that describes the different types of Unicode
37 * BOMs.
38 */
39 public static final class BOM {
40
41 final byte bytes[];
42 private final String description;
43 /**
44 * NONE.
45 */
46 public static final BOM NONE = new BOM(new byte[] {}, "NONE");
47
48 /**
49 * UTF-8 BOM (EF BB BF).
50 */
51 public static final BOM UTF_8 = new BOM(new byte[] { (byte) 0xEF, (byte) 0xBB, (byte) 0xBF }, "UTF-8");
52
53 /**
54 * UTF-16, little-endian (FF FE).
55 */
56 public static final BOM UTF_16_LE = new BOM(new byte[] { (byte) 0xFF, (byte) 0xFE }, "UTF-16 little-endian");
57
58 /**
59 * UTF-16, big-endian (FE FF).
60 */
61 public static final BOM UTF_16_BE = new BOM(new byte[] { (byte) 0xFE, (byte) 0xFF }, "UTF-16 big-endian");
62
63 /**
64 * UTF-32, little-endian (FF FE 00 00).
65 */
66 public static final BOM UTF_32_LE = new BOM(new byte[] { (byte) 0xFF, (byte) 0xFE, (byte) 0x00, (byte) 0x00 },
67 "UTF-32 little-endian");
68
69 /**
70 * UTF-32, big-endian (00 00 FE FF).
71 */
72 public static final BOM UTF_32_BE = new BOM(new byte[] { (byte) 0x00, (byte) 0x00, (byte) 0xFE, (byte) 0xFF },
73 "UTF-32 big-endian");
74
75 /**
76 * Returns a <code>String</code> representation of this <code>BOM</code>
77 * value.
78 */
79 public final String toString() {
80 return description;
81 }
82
83 /**
84 * Returns the bytes corresponding to this <code>BOM</code> value.
85 * @return the bytes corresponding to this <code>BOM</code> value.
86 */
87 public final byte[] getBytes() {
88 final int length = bytes.length;
89 final byte[] result = new byte[length];
90
91 // make a defensive copy
92 System.arraycopy(bytes, 0, result, 0, length);
93
94 return result;
95 }
96
97 private BOM(final byte bom[], final String description) {
98 assert (bom != null) : "invalid BOM: null is not allowed";
99 assert (description != null) : "invalid description: null is not allowed";
100 assert (description.length() != 0) : "invalid description: empty string is not allowed";
101
102 this.bytes = bom;
103 this.description = description;
104 }
105
106 } // BOM
107
108 private final PushbackInputStream in;
109 private final BOM bom;
110 private boolean skipped = false;
111
112
113 /**
114 * Constructs a new <code>UnicodeBOMInputStream</code> that wraps the
115 * specified <code>InputStream</code>. By default skip BOM bytes
116 *
117 * @param inputStream an <code>InputStream</code>.
118 *
119 * @throws NullPointerException when <code>inputStream</code> is
120 * <code>null</code>.
121 * @throws IOException on reading from the specified <code>InputStream</code>
122 * when trying to detect the Unicode BOM.
123 */
124
125 public UnicodeBOMInputStream(final InputStream inputStream) throws IOException {
126 this(inputStream, true);
127 }
128
129
130 /**
131 * Constructs a new <code>UnicodeBOMInputStream</code> that wraps the
132 * specified <code>InputStream</code>.
133 *
134 * @param inputStream an <code>InputStream</code>.
135 * @param skipIfFound to automatically skip BOM bytes if found
136 *
137 * @throws NullPointerException when <code>inputStream</code> is
138 * <code>null</code>.
139 * @throws IOException on reading from the specified <code>InputStream</code>
140 * when trying to detect the Unicode BOM.
141 */
142 public UnicodeBOMInputStream(final InputStream inputStream,
143 boolean skipIfFound) throws IOException {
144 if (inputStream == null) {
145 throw new NullPointerException(
146 "invalid input stream: null is not allowed");
147 }
148 in = new PushbackInputStream(inputStream, 4);
149
150 final byte bom[] = new byte[4];
151 final int read = in.read(bom);
152
153 switch (read) {
154 case 4:
155 if ((bom[0] == (byte) 0xFF) && (bom[1] == (byte) 0xFE)
156 && (bom[2] == (byte) 0x00) && (bom[3] == (byte) 0x00)) {
157 this.bom = BOM.UTF_32_LE;
158 break;
159 } else if ((bom[0] == (byte) 0x00) && (bom[1] == (byte) 0x00)
160 && (bom[2] == (byte) 0xFE) && (bom[3] == (byte) 0xFF)) {
161 this.bom = BOM.UTF_32_BE;
162 break;
163 }
164
165 case 3:
166 if ((bom[0] == (byte) 0xEF) && (bom[1] == (byte) 0xBB)
167 && (bom[2] == (byte) 0xBF)) {
168 this.bom = BOM.UTF_8;
169 break;
170 }
171
172 case 2:
173 if ((bom[0] == (byte) 0xFF) && (bom[1] == (byte) 0xFE)) {
174 this.bom = BOM.UTF_16_LE;
175 break;
176 } else if ((bom[0] == (byte) 0xFE) && (bom[1] == (byte) 0xFF)) {
177 this.bom = BOM.UTF_16_BE;
178 break;
179 }
180
181 default:
182 this.bom = BOM.NONE;
183 break;
184 }
185
186 if (read > 0) {
187 in.unread(bom, 0, read);
188 }
189 if (skipIfFound) {
190 this.skipBOM();
191 }
192
193 }
194
195 /**
196 * Returns the <code>BOM</code> that was detected in the wrapped
197 * <code>InputStream</code> object.
198 *
199 * @return a <code>BOM</code> value.
200 */
201 public final BOM getBOM() {
202 // BOM type is immutable.
203 return bom;
204 }
205
206 /**
207 * Skips the <code>BOM</code> that was found in the wrapped
208 * <code>InputStream</code> object.
209 *
210 * @return this <code>UnicodeBOMInputStream</code>.
211 *
212 * @throws IOException when trying to skip the BOM from the wrapped
213 * <code>InputStream</code> object.
214 */
215 public final synchronized UnicodeBOMInputStream skipBOM()
216 throws IOException {
217 if (!skipped) {
218 long bytesToSkip = bom.bytes.length;
219 long bytesSkipped = in.skip(bytesToSkip);
220 for (long i = bytesSkipped; i < bytesToSkip; i++) {
221 in.read();
222 }
223 skipped = true;
224 }
225 return this;
226 }
227
228 /**
229 * {@inheritDoc}
230 */
231 public int read() throws IOException {
232 this.skipped = true;
233 return in.read();
234 }
235
236 /**
237 * {@inheritDoc}
238 */
239 public int read(final byte b[]) throws IOException {
240 this.skipped = true;
241 return in.read(b, 0, b.length);
242 }
243
244 /**
245 * {@inheritDoc}
246 */
247 public int read(final byte b[], final int off, final int len) throws IOException {
248 this.skipped = true;
249 return in.read(b, off, len);
250 }
251
252 /**
253 * {@inheritDoc}
254 */
255 public long skip(final long n) throws IOException {
256 this.skipped = true;
257 return in.skip(n);
258 }
259
260 /**
261 * {@inheritDoc}
262 */
263 public int available() throws IOException {
264 return in.available();
265 }
266
267 /**
268 * {@inheritDoc}
269 */
270 public void close() throws IOException {
271 in.close();
272 }
273
274 /**
275 * {@inheritDoc}
276 */
277 public synchronized void mark(final int readlimit) {
278 in.mark(readlimit);
279 }
280
281 /**
282 * {@inheritDoc}
283 */
284 public synchronized void reset() throws IOException {
285 in.reset();
286 }
287
288 /**
289 * {@inheritDoc}
290 */
291 public boolean markSupported() {
292 return in.markSupported();
293 }
294
295
296 }
+392
-0
src/main/java/org/mozilla/universalchardet/UniversalDetector.java less more
0 /*
1 (C) Copyright 2016-2017 Alberto Fernández <infjaf@gmail.com>
2 (C) Copyright 2006-2007 Kohei TAKETA <k-tak@void.in> (Java port)
3 (C) Copyright 2001 Netscape Communications Corporation.
4
5 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
6 *
7 * The contents of this file are subject to the Mozilla Public License Version
8 * 1.1 (the "License"); you may not use this file except in compliance with
9 * the License. You may obtain a copy of the License at
10 * http://www.mozilla.org/MPL/
11 *
12 * Software distributed under the License is distributed on an "AS IS" basis,
13 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
14 * for the specific language governing rights and limitations under the
15 * License.
16 *
17 * The Original Code is Mozilla Universal charset detector code.
18 *
19 * The Initial Developer of the Original Code is
20 * Netscape Communications Corporation.
21 * Portions created by the Initial Developer are Copyright (C) 2001
22 * the Initial Developer. All Rights Reserved.
23 *
24 * Contributor(s):
25 * Shy Shalom <shooshX@gmail.com>
26 * Kohei TAKETA <k-tak@void.in> (Java port)
27 *
28 * Alternatively, the contents of this file may be used under the terms of
29 * either the GNU General Public License Version 2 or later (the "GPL"), or
30 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
31 * in which case the provisions of the GPL or the LGPL are applicable instead
32 * of those above. If you wish to allow use of your version of this file only
33 * under the terms of either the GPL or the LGPL, and not to allow others to
34 * use your version of this file under the terms of the MPL, indicate your
35 * decision by deleting the provisions above and replace them with the notice
36 * and other provisions required by the GPL or the LGPL. If you do not delete
37 * the provisions above, a recipient may use your version of this file under
38 * the terms of any one of the MPL, the GPL or the LGPL.
39 *
40 *
41 */
42
43 package org.mozilla.universalchardet;
44
45 import static org.mozilla.universalchardet.Constants.CHARSET_US_ASCCI;
46 import static org.mozilla.universalchardet.Constants.CHARSET_UTF_16BE;
47 import static org.mozilla.universalchardet.Constants.CHARSET_UTF_16LE;
48 import static org.mozilla.universalchardet.Constants.CHARSET_UTF_32BE;
49 import static org.mozilla.universalchardet.Constants.CHARSET_UTF_32LE;
50 import static org.mozilla.universalchardet.Constants.CHARSET_UTF_8;
51 import static org.mozilla.universalchardet.Constants.CHARSET_X_ISO_10646_UCS_4_2143;
52 import static org.mozilla.universalchardet.Constants.CHARSET_X_ISO_10646_UCS_4_3412;
53
54 import java.io.BufferedInputStream;
55 import java.io.File;
56 import java.io.IOException;
57 import java.io.InputStream;
58 import java.nio.file.Files;
59 import java.nio.file.Path;
60
61 import org.mozilla.universalchardet.prober.CharsetProber;
62 import org.mozilla.universalchardet.prober.EscCharsetProber;
63 import org.mozilla.universalchardet.prober.Latin1Prober;
64 import org.mozilla.universalchardet.prober.MBCSGroupProber;
65 import org.mozilla.universalchardet.prober.SBCSGroupProber;
66
67 public class UniversalDetector {
68 ////////////////////////////////////////////////////////////////
69 // constants
70 ////////////////////////////////////////////////////////////////
71 public static final float SHORTCUT_THRESHOLD = 0.95f;
72 public static final float MINIMUM_THRESHOLD = 0.20f;
73
74
75 ////////////////////////////////////////////////////////////////
76 // inner types
77 ////////////////////////////////////////////////////////////////
78 public enum InputState {
79 PURE_ASCII, ESC_ASCII, HIGHBYTE
80 }
81
82
83 ////////////////////////////////////////////////////////////////
84 // fields
85 ////////////////////////////////////////////////////////////////
86 private InputState inputState;
87 private boolean done;
88 private boolean start;
89 private boolean gotData;
90 private boolean onlyPrintableASCII = true;
91 private byte lastChar;
92 private String detectedCharset;
93
94 private CharsetProber[] probers;
95 private CharsetProber escCharsetProber;
96
97 private CharsetListener listener;
98
99
100 ////////////////////////////////////////////////////////////////
101 // methods
102 ////////////////////////////////////////////////////////////////
103
104 public UniversalDetector() {
105 this(null);
106 }
107 /**
108 * @param listener a listener object that is notified of
109 * the detected encocoding. Can be null.
110 */
111 public UniversalDetector(CharsetListener listener) {
112 this.listener = listener;
113 this.escCharsetProber = null;
114 this.probers = new CharsetProber[3];
115
116 reset();
117 }
118
119 public boolean isDone() {
120 return this.done;
121 }
122
123 /**
124 * @return The detected encoding is returned. If the detector couldn't
125 * determine what encoding was used, null is returned.
126 */
127 public String getDetectedCharset() {
128 return this.detectedCharset;
129 }
130
131 public void setListener(CharsetListener listener) {
132 this.listener = listener;
133 }
134
135 public CharsetListener getListener() {
136 return this.listener;
137 }
138 /**
139 * Feed the detector with more data
140 * @param buf The buffer containing the data
141 */
142 public void handleData(final byte[] buf) {
143 handleData(buf, 0, buf.length);
144 }
145 /**
146 * Feed the detector with more data
147 * @param buf Buffer with the data
148 * @param offset initial position of data in buf
149 * @param length length of data
150 */
151 public void handleData(final byte[] buf, int offset, int length) {
152 if (this.done) {
153 return;
154 }
155
156 if (length > 0) {
157 this.gotData = true;
158 }
159
160 if (this.start) {
161 this.start = false;
162 if (length > 3) {
163 String detectedBOM = detectCharsetFromBOM(buf, offset);
164 if (detectedBOM != null) {
165 this.detectedCharset = detectedBOM;
166 this.done = true;
167 return;
168 }
169 }
170 } // if (start) end
171
172 int maxPos = offset + length;
173 for (int i=offset; i<maxPos; ++i) {
174 int c = buf[i] & 0xFF;
175 if ((c & 0x80) != 0 && c != 0xA0) {
176 if (this.inputState != InputState.HIGHBYTE) {
177 this.inputState = InputState.HIGHBYTE;
178
179 if (this.escCharsetProber != null) {
180 this.escCharsetProber = null;
181 }
182
183 if (this.probers[0] == null) {
184 this.probers[0] = new MBCSGroupProber();
185 }
186 if (this.probers[1] == null) {
187 this.probers[1] = new SBCSGroupProber();
188 }
189 if (this.probers[2] == null) {
190 this.probers[2] = new Latin1Prober();
191 }
192 }
193 } else {
194 if (this.inputState == InputState.PURE_ASCII &&
195 (c == 0x1B || (c == 0x7B && this.lastChar == 0x7E))) {
196 this.inputState = InputState.ESC_ASCII;
197 }
198 if (this.inputState == InputState.PURE_ASCII && onlyPrintableASCII) {
199 onlyPrintableASCII =
200 (c >= 0x20 && c <= 0x7e) // Printable characters
201 || c == 0x0A // New Line
202 || c == 0x0D // Carriage return
203 || c== 0x09; // TAB
204 }
205 this.lastChar = buf[i];
206 }
207 } // for end
208
209 CharsetProber.ProbingState st;
210 if (this.inputState == InputState.ESC_ASCII) {
211 if (this.escCharsetProber == null) {
212 this.escCharsetProber = new EscCharsetProber();
213 }
214 st = this.escCharsetProber.handleData(buf, offset, length);
215 if (st == CharsetProber.ProbingState.FOUND_IT) {
216 this.done = true;
217 this.detectedCharset = this.escCharsetProber.getCharSetName();
218 }
219 } else if (this.inputState == InputState.HIGHBYTE) {
220 for (int i=0; i<this.probers.length; ++i) {
221 st = this.probers[i].handleData(buf, offset, length);
222 if (st == CharsetProber.ProbingState.FOUND_IT) {
223 this.done = true;
224 this.detectedCharset = this.probers[i].getCharSetName();
225 return;
226 }
227 }
228 } else { // pure ascii
229 // do nothing
230 }
231 }
232
233 public static String detectCharsetFromBOM(final byte[] buf) {
234 return detectCharsetFromBOM(buf, 0);
235 }
236
237 private static String detectCharsetFromBOM(final byte[] buf, int offset) {
238 if (buf.length > (offset + 3)) {
239 int b1 = buf[offset] & 0xFF;
240 int b2 = buf[offset+1] & 0xFF;
241 int b3 = buf[offset+2] & 0xFF;
242 int b4 = buf[offset+3] & 0xFF;
243
244 switch (b1) {
245 case 0xEF:
246 if (b2 == 0xBB && b3 == 0xBF) {
247 return CHARSET_UTF_8;
248 }
249 break;
250 case 0xFE:
251 if (b2 == 0xFF && b3 == 0x00 && b4 == 0x00) {
252 return CHARSET_X_ISO_10646_UCS_4_3412;
253 } else if (b2 == 0xFF) {
254 return CHARSET_UTF_16BE;
255 }
256 break;
257 case 0x00:
258 if (b2 == 0x00 && b3 == 0xFE && b4 == 0xFF) {
259 return CHARSET_UTF_32BE;
260 } else if (b2 == 0x00 && b3 == 0xFF && b4 == 0xFE) {
261 return CHARSET_X_ISO_10646_UCS_4_2143;
262 }
263 break;
264 case 0xFF:
265 if (b2 == 0xFE && b3 == 0x00 && b4 == 0x00) {
266 return CHARSET_UTF_32LE;
267 } else if (b2 == 0xFE) {
268 return CHARSET_UTF_16LE;
269 }
270 break;
271 default:
272 break;
273 } // swich end
274 }
275 return null;
276 }
277 /**
278 * Marks end of data reading. Finish calculations.
279 */
280 public void dataEnd() {
281 if (!this.gotData) {
282 return;
283 }
284
285 if (this.detectedCharset != null) {
286 this.done = true;
287 if (this.listener != null) {
288 this.listener.report(this.detectedCharset);
289 }
290 return;
291 }
292
293 if (this.inputState == InputState.HIGHBYTE) {
294 float proberConfidence;
295 float maxProberConfidence = 0.0f;
296 int maxProber = 0;
297
298 for (int i=0; i<this.probers.length; ++i) {
299 proberConfidence = this.probers[i].getConfidence();
300 if (proberConfidence > maxProberConfidence) {
301 maxProberConfidence = proberConfidence;
302 maxProber = i;
303 }
304 }
305
306 if (maxProberConfidence > MINIMUM_THRESHOLD) {
307 this.detectedCharset = this.probers[maxProber].getCharSetName();
308 if (this.listener != null) {
309 this.listener.report(this.detectedCharset);
310 }
311 }
312 } else if (this.inputState == InputState.ESC_ASCII) {
313 // do nothing
314 } else if (this.inputState == InputState.PURE_ASCII && this.onlyPrintableASCII) {
315 this.detectedCharset = CHARSET_US_ASCCI;
316 }
317 else {
318 // do nothing
319 }
320 }
321
322 /**
323 * Resets detector to be used again.
324 */
325 public final void reset() {
326 this.done = false;
327 this.start = true;
328 this.detectedCharset = null;
329 this.gotData = false;
330 this.inputState = InputState.PURE_ASCII;
331 this.lastChar = 0;
332
333 if (this.escCharsetProber != null) {
334 this.escCharsetProber.reset();
335 }
336
337 for (int i=0; i<this.probers.length; ++i) {
338 if (this.probers[i] != null) {
339 this.probers[i].reset();
340 }
341 }
342 }
343
344 /**
345 * Gets the charset of a File.
346 *
347 * @param file The file to check charset for
348 * @return The charset of the file, null if cannot be determined
349 * @throws IOException if some IO error occurs
350 */
351 public static String detectCharset(File file) throws IOException {
352 return detectCharset(file.toPath());
353 }
354
355 /**
356 * Gets the charset of a Path.
357 *
358 * @param path The path to file to check charset for
359 * @return The charset of the file, null if cannot be determined
360 * @throws IOException if some IO error occurs
361 */
362 public static String detectCharset(Path path) throws IOException {
363 try (InputStream fis = new BufferedInputStream(Files.newInputStream(path))) {
364 return detectCharset(fis);
365 }
366 }
367
368 /**
369 * Gets the charset of content from InputStream.
370 *
371 * @param inputStream InputStream containing text file
372 * @return The charset of the file, null if cannot be determined
373 * @throws IOException if some IO error occurs
374 */
375 public static String detectCharset(InputStream inputStream) throws IOException {
376 byte[] buf = new byte[4096];
377
378 UniversalDetector detector = new UniversalDetector(null);
379
380 int nread;
381 while ((nread = inputStream.read(buf)) > 0 && !detector.isDone()) {
382 detector.handleData(buf, 0, nread);
383 }
384 detector.dataEnd();
385
386 String encoding = detector.getDetectedCharset();
387 detector.reset();
388 return encoding;
389 }
390
391 }
+137
-0
src/main/java/org/mozilla/universalchardet/prober/Big5Prober.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 * Lersh99
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import org.mozilla.universalchardet.prober.distributionanalysis.Big5DistributionAnalysis;
41 import org.mozilla.universalchardet.prober.statemachine.Big5SMModel;
42 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
43 import org.mozilla.universalchardet.prober.statemachine.SMModel;
44 import org.mozilla.universalchardet.Constants;
45
46
47 public class Big5Prober extends CharsetProber {
48 ////////////////////////////////////////////////////////////////
49 // fields
50 ////////////////////////////////////////////////////////////////
51 private CodingStateMachine codingSM;
52 private ProbingState state;
53
54 private Big5DistributionAnalysis distributionAnalyzer;
55
56 private byte[] lastChar;
57
58 private static final SMModel smModel = new Big5SMModel();
59
60
61 ////////////////////////////////////////////////////////////////
62 // methods
63 ////////////////////////////////////////////////////////////////
64 public Big5Prober() {
65 super();
66 this.codingSM = new CodingStateMachine(smModel);
67 this.distributionAnalyzer = new Big5DistributionAnalysis();
68 this.lastChar = new byte[2];
69 reset();
70 }
71
72 @Override
73 public String getCharSetName() {
74 return Constants.CHARSET_BIG5;
75 }
76
77 @Override
78 public float getConfidence() {
79 return this.distributionAnalyzer.getConfidence();
80 }
81
82 @Override
83 public ProbingState getState() {
84 return this.state;
85 }
86
87 @Override
88 public ProbingState handleData(byte[] buf, int offset, int length) {
89 int codingState;
90
91 int maxPos = offset + length;
92 for (int i=offset; i<maxPos; ++i) {
93 codingState = this.codingSM.nextState(buf[i]);
94 if (codingState == SMModel.ERROR) {
95 this.state = ProbingState.NOT_ME;
96 break;
97 }
98 if (codingState == SMModel.ITSME) {
99 this.state = ProbingState.FOUND_IT;
100 break;
101 }
102 if (codingState == SMModel.START) {
103 int charLen = this.codingSM.getCurrentCharLen();
104 if (i == offset) {
105 this.lastChar[1] = buf[offset];
106 this.distributionAnalyzer.handleOneChar(this.lastChar, 0, charLen);
107 } else {
108 this.distributionAnalyzer.handleOneChar(buf, i-1, charLen);
109 }
110 }
111 }
112
113 this.lastChar[0] = buf[maxPos-1];
114
115 if (this.state == ProbingState.DETECTING) {
116 if (this.distributionAnalyzer.gotEnoughData() && getConfidence() > SHORTCUT_THRESHOLD) {
117 this.state = ProbingState.FOUND_IT;
118 }
119 }
120
121 return this.state;
122 }
123
124 @Override
125 public final void reset() {
126 this.codingSM.reset();
127 this.state = ProbingState.DETECTING;
128 this.distributionAnalyzer.reset();
129 java.util.Arrays.fill(this.lastChar, (byte)0);
130 }
131
132 @Override
133 public void setOption() {
134 //
135 }
136 }
+181
-0
src/main/java/org/mozilla/universalchardet/prober/CharsetProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei TAKETA <k-tak@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import java.nio.ByteBuffer;
41
42
43 public abstract class CharsetProber {
44 ////////////////////////////////////////////////////////////////
45 // constants
46 ////////////////////////////////////////////////////////////////
47 public static final float SHORTCUT_THRESHOLD = 0.95f;
48 public static final int ASCII_A = 0x61; // 'a'
49 public static final int ASCII_Z = 0x7A; // 'z'
50 public static final int ASCII_A_CAPITAL = 0x41; // 'A'
51 public static final int ASCII_Z_CAPITAL = 0x5A; // 'Z'
52 public static final int ASCII_LT = 0x3C; // '<'
53 public static final int ASCII_GT = 0x3E; // '>'
54 public static final int ASCII_SP = 0x20; // ' '
55
56 private boolean active = true;
57
58 ////////////////////////////////////////////////////////////////
59 // inner types
60 ////////////////////////////////////////////////////////////////
61 public enum ProbingState {
62 DETECTING, FOUND_IT, NOT_ME
63 }
64
65
66 ////////////////////////////////////////////////////////////////
67 // methods
68 ////////////////////////////////////////////////////////////////
69 public CharsetProber() {
70 super();
71 }
72
73 public abstract String getCharSetName();
74 public abstract ProbingState handleData(final byte[] buf, int offset, int length);
75 public abstract ProbingState getState();
76 public abstract void reset();
77 public abstract float getConfidence();
78 public abstract void setOption();
79
80 // ByteBuffer.position() indicates number of bytes written.
81 public ByteBuffer filterWithoutEnglishLetters(final byte[] buf, int offset, int length) {
82 ByteBuffer out = ByteBuffer.allocate(length);
83
84 boolean meetMSB = false;
85 byte c;
86
87 int prevPtr = offset;
88 int curPtr = offset;
89 int maxPtr = offset + length;
90
91 for (; curPtr<maxPtr; ++curPtr) {
92 c = buf[curPtr];
93 if (!isAscii(c)) {
94 meetMSB = true;
95 } else if (isAsciiSymbol(c)) {
96 // current char is a symbol, most likely a punctuation.
97 // we treat it as segment delimiter
98 if (meetMSB && curPtr > prevPtr) {
99 // this segment contains more than single symbol,
100 // and it has upper ASCII, we need to keep it
101 out.put(buf, prevPtr, (curPtr-prevPtr));
102 out.put((byte)ASCII_SP);
103 prevPtr = curPtr + 1;
104 meetMSB = false;
105 } else {
106 // ignore current segment.
107 // (either because it is just a symbol or just an English word)
108 prevPtr = curPtr + 1;
109 }
110 }
111 }
112
113 if (meetMSB && curPtr > prevPtr) {
114 out.put(buf, prevPtr, (curPtr-prevPtr));
115 }
116
117 return out;
118 }
119
120 public ByteBuffer filterWithEnglishLetters(final byte[] buf, int offset, int length) {
121 ByteBuffer out = ByteBuffer.allocate(length);
122
123 boolean isInTag = false;
124 byte c;
125
126 int prevPtr = offset;
127 int curPtr = offset;
128 int maxPtr = offset + length;
129
130 for (; curPtr < maxPtr; ++curPtr) {
131 c = buf[curPtr];
132
133 if (c == ASCII_GT) {
134 isInTag = false;
135 } else if (c == ASCII_LT) {
136 isInTag = true;
137 }
138
139 if (isAscii(c) && isAsciiSymbol(c)) {
140 if (curPtr > prevPtr && !isInTag) {
141 // Current segment contains more than just a symbol
142 // and it is not inside a tag, keep it.
143 out.put(buf, prevPtr, (curPtr-prevPtr));
144 out.put((byte)ASCII_SP);
145 prevPtr = curPtr + 1;
146 } else {
147 prevPtr = curPtr + 1;
148 }
149 }
150 }
151
152 // If the current segment contains more than just a symbol
153 // and it is not inside a tag then keep it.
154 if (!isInTag && curPtr > prevPtr) {
155 out.put(buf, prevPtr, (curPtr-prevPtr));
156 }
157
158 return out;
159 }
160
161 private boolean isAscii(byte b) {
162 return ((b & 0x80) == 0);
163 }
164
165 // b must be in ASCII code range (MSB can't be 1).
166 private boolean isAsciiSymbol(byte b) {
167 int c = b & 0xFF;
168 return ((c < ASCII_A_CAPITAL) ||
169 (c > ASCII_Z_CAPITAL && c < ASCII_A) ||
170 (c > ASCII_Z));
171 }
172
173 public boolean isActive() {
174 return active;
175 }
176
177 public void setActive(boolean active) {
178 this.active = active;
179 }
180 }
+145
-0
src/main/java/org/mozilla/universalchardet/prober/EUCJPProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
40 import org.mozilla.universalchardet.prober.statemachine.EUCJPSMModel;
41 import org.mozilla.universalchardet.prober.statemachine.SMModel;
42 import org.mozilla.universalchardet.prober.contextanalysis.EUCJPContextAnalysis;
43 import org.mozilla.universalchardet.prober.distributionanalysis.EUCJPDistributionAnalysis;
44 import org.mozilla.universalchardet.Constants;
45
46
47 public class EUCJPProber extends CharsetProber {
48 ////////////////////////////////////////////////////////////////
49 // fields
50 ////////////////////////////////////////////////////////////////
51 private CodingStateMachine codingSM;
52 private ProbingState state;
53
54 private EUCJPContextAnalysis contextAnalyzer;
55 private EUCJPDistributionAnalysis distributionAnalyzer;
56
57 private byte[] lastChar;
58
59 private static final SMModel smModel = new EUCJPSMModel();
60
61
62 ////////////////////////////////////////////////////////////////
63 // methods
64 ////////////////////////////////////////////////////////////////
65 public EUCJPProber() {
66 super();
67 this.codingSM = new CodingStateMachine(smModel);
68 this.contextAnalyzer = new EUCJPContextAnalysis();
69 this.distributionAnalyzer = new EUCJPDistributionAnalysis();
70 this.lastChar = new byte[2];
71 reset();
72 }
73
74 @Override
75 public String getCharSetName() {
76 return Constants.CHARSET_EUC_JP;
77 }
78
79 @Override
80 public float getConfidence() {
81 float contextCf = this.contextAnalyzer.getConfidence();
82 float distribCf = this.distributionAnalyzer.getConfidence();
83
84 return Math.max(contextCf, distribCf);
85 }
86
87 @Override
88 public ProbingState getState() {
89 return this.state;
90 }
91
92 @Override
93 public ProbingState handleData(byte[] buf, int offset, int length) {
94 int codingState;
95
96 int maxPos = offset + length;
97 for (int i=offset; i<maxPos; ++i) {
98 codingState = this.codingSM.nextState(buf[i]);
99 if (codingState == SMModel.ERROR) {
100 this.state = ProbingState.NOT_ME;
101 break;
102 }
103 if (codingState == SMModel.ITSME) {
104 this.state = ProbingState.FOUND_IT;
105 break;
106 }
107 if (codingState == SMModel.START) {
108 int charLen = this.codingSM.getCurrentCharLen();
109
110 if (i == offset) {
111 this.lastChar[1] = buf[offset];
112 this.contextAnalyzer.handleOneChar(this.lastChar, 0, charLen);
113 this.distributionAnalyzer.handleOneChar(this.lastChar, 0, charLen);
114 } else {
115 this.contextAnalyzer.handleOneChar(buf, i-1, charLen);
116 this.distributionAnalyzer.handleOneChar(buf, i-1, charLen);
117 }
118 }
119 }
120
121 this.lastChar[0] = buf[maxPos-1];
122
123 if (this.state == ProbingState.DETECTING) {
124 if (this.contextAnalyzer.gotEnoughData() && getConfidence() > SHORTCUT_THRESHOLD) {
125 this.state = ProbingState.FOUND_IT;
126 }
127 }
128
129 return this.state;
130 }
131
132 @Override
133 public final void reset() {
134 this.codingSM.reset();
135 this.state = ProbingState.DETECTING;
136 this.contextAnalyzer.reset();
137 this.distributionAnalyzer.reset();
138 java.util.Arrays.fill(this.lastChar, (byte)0);
139 }
140
141 @Override
142 public void setOption() {
143 }
144 }
+135
-0
src/main/java/org/mozilla/universalchardet/prober/EUCKRProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.prober.distributionanalysis.EUCKRDistributionAnalysis;
40 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
41 import org.mozilla.universalchardet.prober.statemachine.EUCKRSMModel;
42 import org.mozilla.universalchardet.prober.statemachine.SMModel;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class EUCKRProber extends CharsetProber {
47 ////////////////////////////////////////////////////////////////
48 // fields
49 ////////////////////////////////////////////////////////////////
50 private CodingStateMachine codingSM;
51 private ProbingState state;
52
53 private EUCKRDistributionAnalysis distributionAnalyzer;
54
55 private byte[] lastChar;
56
57 private static final SMModel smModel = new EUCKRSMModel();
58
59
60 ////////////////////////////////////////////////////////////////
61 // methods
62 ////////////////////////////////////////////////////////////////
63 public EUCKRProber() {
64 super();
65 this.codingSM = new CodingStateMachine(smModel);
66 this.distributionAnalyzer = new EUCKRDistributionAnalysis();
67 this.lastChar = new byte[2];
68 reset();
69 }
70
71 @Override
72 public String getCharSetName() {
73 return Constants.CHARSET_EUC_KR;
74 }
75
76 @Override
77 public float getConfidence() {
78 return this.distributionAnalyzer.getConfidence();
79 }
80
81 @Override
82 public ProbingState getState() {
83 return this.state;
84 }
85
86 @Override
87 public ProbingState handleData(byte[] buf, int offset, int length) {
88 int codingState;
89
90 int maxPos = offset + length;
91 for (int i=offset; i<maxPos; ++i) {
92 codingState = codingSM.nextState(buf[i]);
93 if (codingState == SMModel.ERROR) {
94 this.state = ProbingState.NOT_ME;
95 break;
96 }
97 if (codingState == SMModel.ITSME) {
98 this.state = ProbingState.FOUND_IT;
99 break;
100 }
101 if (codingState == SMModel.START) {
102 int charLen = this.codingSM.getCurrentCharLen();
103 if (i == offset) {
104 this.lastChar[1] = buf[offset];
105 this.distributionAnalyzer.handleOneChar(this.lastChar, 0, charLen);
106 } else {
107 this.distributionAnalyzer.handleOneChar(buf, i-1, charLen);
108 }
109 }
110 }
111
112 this.lastChar[0] = buf[maxPos-1];
113
114 if (this.state == ProbingState.DETECTING) {
115 if (this.distributionAnalyzer.gotEnoughData() && getConfidence() > SHORTCUT_THRESHOLD) {
116 this.state = ProbingState.FOUND_IT;
117 }
118 }
119
120 return this.state;
121 }
122
123 @Override
124 public final void reset() {
125 this.codingSM.reset();
126 this.state = ProbingState.DETECTING;
127 this.distributionAnalyzer.reset();
128 java.util.Arrays.fill(this.lastChar, (byte)0);
129 }
130
131 @Override
132 public void setOption() {
133 }
134 }
+136
-0
src/main/java/org/mozilla/universalchardet/prober/EUCTWProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 * Lersh99
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import org.mozilla.universalchardet.prober.distributionanalysis.EUCTWDistributionAnalysis;
41 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
42 import org.mozilla.universalchardet.prober.statemachine.EUCTWSMModel;
43 import org.mozilla.universalchardet.prober.statemachine.SMModel;
44 import org.mozilla.universalchardet.Constants;
45
46
47 public class EUCTWProber extends CharsetProber {
48 ////////////////////////////////////////////////////////////////
49 // fields
50 ////////////////////////////////////////////////////////////////
51 private CodingStateMachine codingSM;
52 private ProbingState state;
53
54 private EUCTWDistributionAnalysis distributionAnalyzer;
55
56 private byte[] lastChar;
57
58 private static final SMModel smModel = new EUCTWSMModel();
59
60
61 ////////////////////////////////////////////////////////////////
62 // methods
63 ////////////////////////////////////////////////////////////////
64 public EUCTWProber() {
65 super();
66 this.codingSM = new CodingStateMachine(smModel);
67 this.distributionAnalyzer = new EUCTWDistributionAnalysis();
68 this.lastChar = new byte[2];
69 reset();
70 }
71
72 @Override
73 public String getCharSetName() {
74 return Constants.CHARSET_EUC_TW;
75 }
76
77 @Override
78 public float getConfidence() {
79 return this.distributionAnalyzer.getConfidence();
80 }
81
82 @Override
83 public ProbingState getState() {
84 return this.state;
85 }
86
87 @Override
88 public ProbingState handleData(byte[] buf, int offset, int length) {
89 int codingState;
90
91 int maxPos = offset + length;
92 for (int i=offset; i<maxPos; ++i) {
93 codingState = this.codingSM.nextState(buf[i]);
94 if (codingState == SMModel.ERROR) {
95 this.state = ProbingState.NOT_ME;
96 break;
97 }
98 if (codingState == SMModel.ITSME) {
99 this.state = ProbingState.FOUND_IT;
100 break;
101 }
102 if (codingState == SMModel.START) {
103 int charLen = this.codingSM.getCurrentCharLen();
104 if (i == offset) {
105 this.lastChar[1] = buf[offset];
106 this.distributionAnalyzer.handleOneChar(this.lastChar, 0, charLen);
107 } else {
108 this.distributionAnalyzer.handleOneChar(buf, i-1, charLen);
109 }
110 }
111 }
112
113 this.lastChar[0] = buf[maxPos-1];
114
115 if (this.state == ProbingState.DETECTING) {
116 if (this.distributionAnalyzer.gotEnoughData() && getConfidence() > SHORTCUT_THRESHOLD) {
117 this.state = ProbingState.FOUND_IT;
118 }
119 }
120
121 return this.state;
122 }
123
124 @Override
125 public final void reset() {
126 this.codingSM.reset();
127 this.state = ProbingState.DETECTING;
128 this.distributionAnalyzer.reset();
129 java.util.Arrays.fill(this.lastChar, (byte) 0);
130 }
131
132 @Override
133 public void setOption()
134 {}
135 }
+136
-0
src/main/java/org/mozilla/universalchardet/prober/EscCharsetProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
40 import org.mozilla.universalchardet.prober.statemachine.SMModel;
41 import org.mozilla.universalchardet.prober.statemachine.HZSMModel;
42 import org.mozilla.universalchardet.prober.statemachine.ISO2022CNSMModel;
43 import org.mozilla.universalchardet.prober.statemachine.ISO2022JPSMModel;
44 import org.mozilla.universalchardet.prober.statemachine.ISO2022KRSMModel;
45
46
47 public class EscCharsetProber extends CharsetProber {
48 ////////////////////////////////////////////////////////////////
49 // fields
50 ////////////////////////////////////////////////////////////////
51 private CodingStateMachine[] codingSM;
52 private int activeSM;
53 private ProbingState state;
54 private String detectedCharset;
55
56 private static final HZSMModel hzsModel = new HZSMModel();
57 private static final ISO2022CNSMModel iso2022cnModel = new ISO2022CNSMModel();
58 private static final ISO2022JPSMModel iso2022jpModel = new ISO2022JPSMModel();
59 private static final ISO2022KRSMModel iso2022krModel = new ISO2022KRSMModel();
60
61
62 ////////////////////////////////////////////////////////////////
63 // methods
64 ////////////////////////////////////////////////////////////////
65 public EscCharsetProber() {
66 super();
67
68 this.codingSM = new CodingStateMachine[4];
69 this.codingSM[0] = new CodingStateMachine(hzsModel);
70 this.codingSM[1] = new CodingStateMachine(iso2022cnModel);
71 this.codingSM[2] = new CodingStateMachine(iso2022jpModel);
72 this.codingSM[3] = new CodingStateMachine(iso2022krModel);
73
74 reset();
75 }
76
77 @Override
78 public String getCharSetName() {
79 return this.detectedCharset;
80 }
81
82 @Override
83 public float getConfidence() {
84 return 0.99f;
85 }
86
87 @Override
88 public ProbingState getState() {
89 return this.state;
90 }
91
92 @Override
93 public ProbingState handleData(byte[] buf, int offset, int length) {
94 int codingState;
95
96 int maxPos = offset + length;
97 for (int i=offset; i<maxPos && this.state==ProbingState.DETECTING; ++i) {
98 for (int j=this.activeSM-1; j>=0; --j) {
99 codingState = this.codingSM[j].nextState(buf[i]);
100 if (codingState == SMModel.ERROR) {
101 --this.activeSM;
102 if (this.activeSM <= 0) {
103 this.state = ProbingState.NOT_ME;
104 return this.state;
105 } else if (j != this.activeSM) {
106 CodingStateMachine t;
107 t = this.codingSM[this.activeSM];
108 this.codingSM[this.activeSM] = this.codingSM[j];
109 this.codingSM[j] = t;
110 }
111 } else if (codingState == SMModel.ITSME) {
112 this.state = ProbingState.FOUND_IT;
113 this.detectedCharset = this.codingSM[j].getCodingStateMachine();
114 return this.state;
115 }
116 }
117 }
118
119 return this.state;
120 }
121
122 @Override
123 public final void reset() {
124 this.state = ProbingState.DETECTING;
125 for (int i = 0; i < this.codingSM.length; ++i) {
126 this.codingSM[i].reset();
127 }
128 this.activeSM = this.codingSM.length;
129 this.detectedCharset = null;
130 }
131
132 @Override
133 public void setOption() {
134 }
135 }
+136
-0
src/main/java/org/mozilla/universalchardet/prober/GB18030Prober.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 * Lersh99
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import org.mozilla.universalchardet.prober.distributionanalysis.GB2312DistributionAnalysis;
41 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
42 import org.mozilla.universalchardet.prober.statemachine.GB18030SMModel;
43 import org.mozilla.universalchardet.prober.statemachine.SMModel;
44 import org.mozilla.universalchardet.Constants;
45
46
47 public class GB18030Prober extends CharsetProber {
48 ////////////////////////////////////////////////////////////////
49 // fields
50 ////////////////////////////////////////////////////////////////
51 private CodingStateMachine codingSM;
52 private ProbingState state;
53
54 private GB2312DistributionAnalysis distributionAnalyzer;
55
56 private byte[] lastChar;
57
58 private static final SMModel smModel = new GB18030SMModel();
59
60
61 ////////////////////////////////////////////////////////////////
62 // methods
63 ////////////////////////////////////////////////////////////////
64 public GB18030Prober() {
65 super();
66 this.codingSM = new CodingStateMachine(smModel);
67 this.distributionAnalyzer = new GB2312DistributionAnalysis();
68 this.lastChar = new byte[2];
69 reset();
70 }
71
72 @Override
73 public String getCharSetName() {
74 return Constants.CHARSET_GB18030;
75 }
76
77 @Override
78 public float getConfidence() {
79 return this.distributionAnalyzer.getConfidence();
80 }
81
82 @Override
83 public ProbingState getState() {
84 return this.state;
85 }
86
87 @Override
88 public ProbingState handleData(byte[] buf, int offset, int length) {
89 int codingState;
90
91 int maxPos = offset + length;
92 for (int i=offset; i<maxPos; ++i) {
93 codingState = this.codingSM.nextState(buf[i]);
94 if (codingState == SMModel.ERROR) {
95 this.state = ProbingState.NOT_ME;
96 break;
97 }
98 if (codingState == SMModel.ITSME) {
99 this.state = ProbingState.FOUND_IT;
100 break;
101 }
102 if (codingState == SMModel.START) {
103 int charLen = this.codingSM.getCurrentCharLen();
104 if (i == offset) {
105 this.lastChar[1] = buf[offset];
106 this.distributionAnalyzer.handleOneChar(this.lastChar, 0, charLen);
107 } else {
108 this.distributionAnalyzer.handleOneChar(buf, i-1, charLen);
109 }
110 }
111 }
112
113 this.lastChar[0] = buf[maxPos-1];
114
115 if (this.state == ProbingState.DETECTING) {
116 if (this.distributionAnalyzer.gotEnoughData() && getConfidence() > SHORTCUT_THRESHOLD) {
117 this.state = ProbingState.FOUND_IT;
118 }
119 }
120
121 return this.state;
122 }
123
124 @Override
125 public final void reset() {
126 this.codingSM.reset();
127 this.state = ProbingState.DETECTING;
128 this.distributionAnalyzer.reset();
129 java.util.Arrays.fill(this.lastChar, (byte)0);
130 }
131
132 @Override
133 public void setOption() {
134 }
135 }
+213
-0
src/main/java/org/mozilla/universalchardet/prober/HebrewProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Shy Shalom <shooshX@gmail.com>
17 * Portions created by the Initial Developer are Copyright (C) 2005
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class HebrewProber extends CharsetProber {
43 ////////////////////////////////////////////////////////////////
44 // fields
45 ////////////////////////////////////////////////////////////////
46 public static final int FINAL_KAF = 0xEA;
47 public static final int NORMAL_KAF = 0xEB;
48 public static final int FINAL_MEM = 0xED;
49 public static final int NORMAL_MEM = 0xEE;
50 public static final int FINAL_NUN = 0xEF;
51 public static final int NORMAL_NUN = 0xF0;
52 public static final int FINAL_PE = 0xF3;
53 public static final int NORMAL_PE = 0xF4;
54 public static final int FINAL_TSADI = 0xF5;
55 public static final int NORMAL_TSADI= 0xF6;
56
57 public static final byte SPACE = 0x20;
58
59 public static final int MIN_FINAL_CHAR_DISTANCE = 5;
60 public static final float MIN_MODEL_DISTANCE = 0.01f;
61
62
63 ////////////////////////////////////////////////////////////////
64 // fields
65 ////////////////////////////////////////////////////////////////
66 private int finalCharLogicalScore;
67 private int finalCharVisualScore;
68 private byte prev;
69 private byte beforePrev;
70
71 private CharsetProber logicalProber;
72 private CharsetProber visualProber;
73
74
75 ////////////////////////////////////////////////////////////////
76 // methods
77 ////////////////////////////////////////////////////////////////
78 public HebrewProber() {
79 super();
80 this.logicalProber = null;
81 this.visualProber = null;
82 reset();
83 }
84
85 public void setModalProbers(CharsetProber logicalProber, CharsetProber visualProber) {
86 this.logicalProber = logicalProber;
87 this.visualProber = visualProber;
88 }
89
90 @Override
91 public String getCharSetName() {
92 // If the final letter score distance is dominant enough, rely on it.
93 int finalsub = this.finalCharLogicalScore - this.finalCharVisualScore;
94 if (finalsub >= MIN_FINAL_CHAR_DISTANCE) {
95 return Constants.CHARSET_WINDOWS_1255;
96 }
97 if (finalsub <= -MIN_FINAL_CHAR_DISTANCE) {
98 return Constants.CHARSET_ISO_8859_8;
99 }
100
101 // It's not dominant enough, try to rely on the model scores instead.
102 float modelsub = this.logicalProber.getConfidence() - this.visualProber.getConfidence();
103 if (modelsub > MIN_MODEL_DISTANCE) {
104 return Constants.CHARSET_WINDOWS_1255;
105 }
106 if (modelsub < -MIN_MODEL_DISTANCE) {
107 return Constants.CHARSET_ISO_8859_8;
108 }
109
110 // Still no good, back to final letter distance, maybe it'll save the day.
111 if (finalsub < 0) {
112 return Constants.CHARSET_ISO_8859_8;
113 }
114
115 // (finalsub > 0 - Logical) or (don't know what to do) default to Logical.
116 return Constants.CHARSET_WINDOWS_1255;
117 }
118
119 @Override
120 public float getConfidence() {
121 return 0.0f;
122 }
123
124 @Override
125 public ProbingState getState() {
126 // Remain active as long as any of the model probers are active.
127 if ((this.logicalProber.getState() == ProbingState.NOT_ME) &&
128 (this.visualProber.getState() == ProbingState.NOT_ME)) {
129 return ProbingState.NOT_ME;
130 }
131
132 return ProbingState.DETECTING;
133 }
134
135 @Override
136 public ProbingState handleData(byte[] buf, int offset, int length) {
137 if (getState() == ProbingState.NOT_ME) {
138 return ProbingState.NOT_ME;
139 }
140
141 byte c;
142 int maxPos = offset + length;
143 for (int i=offset; i<maxPos; ++i) {
144 c = buf[i];
145 if (c == SPACE) {
146 if (this.beforePrev != SPACE) {
147 if (isFinal(this.prev)) {
148 ++this.finalCharLogicalScore;
149 } else if (isNonFinal(this.prev)) {
150 ++this.finalCharVisualScore;
151 }
152 }
153 } else {
154 if ((this.beforePrev == SPACE) &&
155 isFinal(this.prev) &&
156 c != SPACE) {
157 ++this.finalCharVisualScore;
158 }
159 }
160 this.beforePrev = this.prev;
161 this.prev = c;
162 }
163
164 return ProbingState.DETECTING;
165 }
166
167 @Override
168 public final void reset() {
169 this.finalCharLogicalScore = 0;
170 this.finalCharVisualScore = 0;
171
172 // mPrev and mBeforePrev are initialized to space in order to simulate a word
173 // delimiter at the beginning of the data
174 this.prev = SPACE;
175 this.beforePrev = SPACE;
176 }
177
178 @Override
179 public void setOption() {
180 }
181
182 protected static boolean isFinal(byte b) {
183 int c = b & 0xFF;
184 return (
185 c == FINAL_KAF ||
186 c == FINAL_MEM ||
187 c == FINAL_NUN ||
188 c == FINAL_PE ||
189 c == FINAL_TSADI
190 );
191 }
192
193 protected static boolean isNonFinal(byte b) {
194 int c = b & 0xFF;
195 return (
196 c == NORMAL_KAF ||
197 c == NORMAL_MEM ||
198 c == NORMAL_NUN ||
199 c == NORMAL_PE
200 );
201 // The normal Tsadi is not a good Non-Final letter due to words like
202 // 'lechotet' (to chat) containing an apostrophe after the tsadi. This
203 // apostrophe is converted to a space in FilterWithoutEnglishLetters causing
204 // the Non-Final tsadi to appear at an end of a word even though this is not
205 // the case in the original text.
206 // The letters Pe and Kaf rarely display a related behavior of not being a
207 // good Non-Final letter. Words like 'Pop', 'Winamp' and 'Mubarak' for
208 // example legally end with a Non-Final Pe or Kaf. However, the benefit of
209 // these letters as Non-Final letters outweighs the damage since these words
210 // are quite rare.
211 }
212 }
+199
-0
src/main/java/org/mozilla/universalchardet/prober/Latin1Prober.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei Taketa <k-tak@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import java.nio.ByteBuffer;
41 import org.mozilla.universalchardet.Constants;
42
43
44 public class Latin1Prober extends CharsetProber {
45 public static final byte UDF = 0;
46 public static final byte OTH = 1;
47 public static final byte ASC = 2;
48 public static final byte ASS = 3;
49 public static final byte ACV = 4;
50 public static final byte ACO = 5;
51 public static final byte ASV = 6;
52 public static final byte ASO = 7;
53 public static final int CLASS_NUM = 8;
54 public static final int FREQ_CAT_NUM = 4;
55
56
57 ////////////////////////////////////////////////////////////////
58 // constants continued
59 ////////////////////////////////////////////////////////////////
60 private static final byte[] latin1CharToClass = new byte[] {
61 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 00 - 07
62 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 08 - 0F
63 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 10 - 17
64 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 18 - 1F
65 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 20 - 27
66 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 28 - 2F
67 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 30 - 37
68 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 38 - 3F
69 OTH, ASC, ASC, ASC, ASC, ASC, ASC, ASC, // 40 - 47
70 ASC, ASC, ASC, ASC, ASC, ASC, ASC, ASC, // 48 - 4F
71 ASC, ASC, ASC, ASC, ASC, ASC, ASC, ASC, // 50 - 57
72 ASC, ASC, ASC, OTH, OTH, OTH, OTH, OTH, // 58 - 5F
73 OTH, ASS, ASS, ASS, ASS, ASS, ASS, ASS, // 60 - 67
74 ASS, ASS, ASS, ASS, ASS, ASS, ASS, ASS, // 68 - 6F
75 ASS, ASS, ASS, ASS, ASS, ASS, ASS, ASS, // 70 - 77
76 ASS, ASS, ASS, OTH, OTH, OTH, OTH, OTH, // 78 - 7F
77 OTH, UDF, OTH, ASO, OTH, OTH, OTH, OTH, // 80 - 87
78 OTH, OTH, ACO, OTH, ACO, UDF, ACO, UDF, // 88 - 8F
79 UDF, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 90 - 97
80 OTH, OTH, ASO, OTH, ASO, UDF, ASO, ACO, // 98 - 9F
81 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // A0 - A7
82 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // A8 - AF
83 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // B0 - B7
84 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // B8 - BF
85 ACV, ACV, ACV, ACV, ACV, ACV, ACO, ACO, // C0 - C7
86 ACV, ACV, ACV, ACV, ACV, ACV, ACV, ACV, // C8 - CF
87 ACO, ACO, ACV, ACV, ACV, ACV, ACV, OTH, // D0 - D7
88 ACV, ACV, ACV, ACV, ACV, ACO, ACO, ACO, // D8 - DF
89 ASV, ASV, ASV, ASV, ASV, ASV, ASO, ASO, // E0 - E7
90 ASV, ASV, ASV, ASV, ASV, ASV, ASV, ASV, // E8 - EF
91 ASO, ASO, ASV, ASV, ASV, ASV, ASV, OTH, // F0 - F7
92 ASV, ASV, ASV, ASV, ASV, ASO, ASO, ASO, // F8 - FF
93 };
94
95 private static final byte[] latin1ClassModel = new byte[] {
96 /* UDF OTH ASC ASS ACV ACO ASV ASO */
97 /*UDF*/ 0, 0, 0, 0, 0, 0, 0, 0,
98 /*OTH*/ 0, 3, 3, 3, 3, 3, 3, 3,
99 /*ASC*/ 0, 3, 3, 3, 3, 3, 3, 3,
100 /*ASS*/ 0, 3, 3, 3, 1, 1, 3, 3,
101 /*ACV*/ 0, 3, 3, 3, 1, 2, 1, 2,
102 /*ACO*/ 0, 3, 3, 3, 3, 3, 3, 3,
103 /*ASV*/ 0, 3, 1, 3, 1, 1, 1, 3,
104 /*ASO*/ 0, 3, 1, 3, 1, 1, 3, 3,
105 };
106
107 private ProbingState state;
108 private byte lastCharClass;
109 private int[] freqCounter;
110
111
112 public Latin1Prober() {
113 super();
114 this.freqCounter = new int[FREQ_CAT_NUM];
115 reset();
116 }
117
118 @Override
119 public String getCharSetName() {
120 return Constants.CHARSET_WINDOWS_1252;
121 }
122
123 @Override
124 public float getConfidence() {
125 if (this.state == ProbingState.NOT_ME) {
126 return 0.01f;
127 }
128
129 float confidence;
130 int total = 0;
131 for (int i = 0; i < this.freqCounter.length; ++i) {
132 total += this.freqCounter[i];
133 }
134
135 if (total <= 0) {
136 confidence = 0.0f;
137 } else {
138 confidence = this.freqCounter[3] * 1.0f / total;
139 confidence -= this.freqCounter[1] * 20.0f / total;
140 }
141
142 if (confidence < 0.0f) {
143 confidence = 0.0f;
144 }
145
146 // lower the confidence of latin1 so that other more accurate detector
147 // can take priority.
148 confidence *= 0.50f;
149
150 return confidence;
151 }
152
153 @Override
154 public ProbingState getState() {
155 return this.state;
156 }
157
158 @Override
159 public ProbingState handleData(byte[] buf, int offset, int length) {
160 ByteBuffer newBufTmp = filterWithEnglishLetters(buf, offset, length);
161
162 byte charClass;
163 byte freq;
164
165 byte[] newBuf = newBufTmp.array();
166 int newBufLen = newBufTmp.position();
167
168 for (int i = 0; i < newBufLen; ++i) {
169 int c = newBuf[i] & 0xFF;
170 charClass = latin1CharToClass[c];
171 freq = latin1ClassModel[this.lastCharClass * CLASS_NUM + charClass];
172 if (freq == 0) {
173 this.state = ProbingState.NOT_ME;
174 break;
175 }
176 ++this.freqCounter[freq];
177 this.lastCharClass = charClass;
178 }
179
180 return this.state;
181 }
182
183 @Override
184 public final void reset() {
185 this.state = ProbingState.DETECTING;
186 this.lastCharClass = OTH;
187 for (int i = 0; i < this.freqCounter.length; ++i) {
188 this.freqCounter[i] = 0;
189 }
190 }
191
192 @Override
193 public void setOption() {
194 }
195
196
197
198 }
+172
-0
src/main/java/org/mozilla/universalchardet/prober/MBCSGroupProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei TAKETA <k-tak@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import java.util.ArrayList;
41 import java.util.List;
42
43 public class MBCSGroupProber extends CharsetProber {
44 ////////////////////////////////////////////////////////////////
45 // fields
46 ////////////////////////////////////////////////////////////////
47 private ProbingState state;
48 private List<CharsetProber> probers = new ArrayList<>();
49 private CharsetProber bestGuess;
50 private int activeNum;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public MBCSGroupProber() {
57 super();
58
59
60 probers.add(new UTF8Prober());
61 probers.add(new SJISProber());
62 probers.add(new EUCJPProber());
63 probers.add(new GB18030Prober());
64 probers.add(new EUCKRProber());
65 probers.add(new Big5Prober());
66 probers.add(new EUCTWProber());
67
68 reset();
69 }
70
71 @Override
72 public String getCharSetName() {
73 if (this.bestGuess == null) {
74 getConfidence();
75 if (this.bestGuess == null) {
76 this.bestGuess = probers.get(0);
77 }
78 }
79 return this.bestGuess.getCharSetName();
80 }
81
82 @Override
83 public float getConfidence() {
84 float bestConf = 0.0f;
85 float cf;
86
87 if (this.state == ProbingState.FOUND_IT) {
88 return 0.99f;
89 } else if (this.state == ProbingState.NOT_ME) {
90 return 0.01f;
91 } else {
92 for(CharsetProber prober: probers) {
93 if (!prober.isActive()) {
94 continue;
95 }
96 cf = prober.getConfidence();
97 if (bestConf < cf) {
98 bestConf = cf;
99 this.bestGuess = prober;
100 }
101 }
102 }
103
104 return bestConf;
105 }
106
107 @Override
108 public ProbingState getState() {
109 return this.state;
110 }
111
112 @Override
113 public ProbingState handleData(byte[] buf, int offset, int length) {
114 ProbingState st;
115
116 boolean keepNext = true;
117 byte[] highbyteBuf = new byte[length];
118 int highpos = 0;
119
120 int maxPos = offset + length;
121 for (int i=offset; i<maxPos; ++i) {
122 if ((buf[i] & 0x80) != 0) {
123 highbyteBuf[highpos++] = buf[i];
124 keepNext = true;
125 } else {
126 //if previous is highbyte, keep this even it is a ASCII
127 if (keepNext) {
128 highbyteBuf[highpos++] = buf[i];
129 keepNext = false;
130 }
131 }
132 }
133
134 for(CharsetProber prober: this.probers) {
135 if (!prober.isActive()) {
136 continue;
137 }
138 st = prober.handleData(highbyteBuf, 0, highpos);
139 if (st == ProbingState.FOUND_IT) {
140 this.bestGuess = prober;
141 this.state = ProbingState.FOUND_IT;
142 break;
143 } else if (st == ProbingState.NOT_ME) {
144 prober.setActive(false);
145 this.activeNum--;
146 if (this.activeNum <= 0) {
147 this.state = ProbingState.NOT_ME;
148 break;
149 }
150 }
151 }
152
153 return this.state;
154 }
155
156 @Override
157 public final void reset() {
158 this.activeNum = 0;
159 for (CharsetProber prober: this.probers) {
160 prober.reset();
161 prober.setActive(true);
162 this.activeNum++;
163 }
164 this.bestGuess = null;
165 this.state = ProbingState.DETECTING;
166 }
167
168 @Override
169 public void setOption()
170 {}
171 }
+183
-0
src/main/java/org/mozilla/universalchardet/prober/SBCSGroupProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei TAKETA <k-tak@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37 package org.mozilla.universalchardet.prober;
38
39 import java.nio.ByteBuffer;
40 import java.util.ArrayList;
41 import java.util.List;
42
43 import org.mozilla.universalchardet.prober.sequence.HebrewModel;
44 import org.mozilla.universalchardet.prober.sequence.Ibm855Model;
45 import org.mozilla.universalchardet.prober.sequence.Ibm866Model;
46 import org.mozilla.universalchardet.prober.sequence.Koi8rModel;
47 import org.mozilla.universalchardet.prober.sequence.Latin5BulgarianModel;
48 import org.mozilla.universalchardet.prober.sequence.Latin5Model;
49 import org.mozilla.universalchardet.prober.sequence.Latin7Model;
50 import org.mozilla.universalchardet.prober.sequence.MacCyrillicModel;
51 import org.mozilla.universalchardet.prober.sequence.SequenceModel;
52 import org.mozilla.universalchardet.prober.sequence.ThaiModel;
53 import org.mozilla.universalchardet.prober.sequence.Win1251BulgarianModel;
54 import org.mozilla.universalchardet.prober.sequence.Win1251Model;
55 import org.mozilla.universalchardet.prober.sequence.Win1253Model;
56
57 public class SBCSGroupProber extends CharsetProber {
58
59 private ProbingState state;
60 private List<CharsetProber> probers = new ArrayList<>();
61 private CharsetProber bestGuess;
62 private int activeNum;
63
64 public SBCSGroupProber() {
65 super();
66
67 probers.add(new SingleByteCharsetProber(new Win1251Model()));
68 probers.add(new SingleByteCharsetProber(new Koi8rModel()));
69 probers.add(new SingleByteCharsetProber(new Latin5Model()));
70 probers.add(new SingleByteCharsetProber(new MacCyrillicModel()));
71 probers.add(new SingleByteCharsetProber(new Ibm866Model()));
72 probers.add(new SingleByteCharsetProber(new Ibm855Model()));
73 probers.add(new SingleByteCharsetProber(new Latin7Model()));
74 probers.add(new SingleByteCharsetProber(new Win1253Model()));
75 probers.add(new SingleByteCharsetProber(new Latin5BulgarianModel()));
76 probers.add(new SingleByteCharsetProber(new Win1251BulgarianModel()));
77 probers.add(new SingleByteCharsetProber(new ThaiModel()));
78
79 SequenceModel hebrewModel = new HebrewModel();
80 HebrewProber hebprober = new HebrewProber();
81 CharsetProber singleByte1 = new SingleByteCharsetProber(hebrewModel, false, hebprober);
82 CharsetProber singleByte2 =new SingleByteCharsetProber(hebrewModel, true, hebprober);
83 hebprober.setModalProbers(singleByte1, singleByte2);
84
85 probers.add(hebprober);
86 probers.add(singleByte1);
87 probers.add(singleByte2);
88
89 reset();
90 }
91
92 @Override
93 public String getCharSetName() {
94 if (this.bestGuess == null) {
95 getConfidence();
96 if (this.bestGuess == null) {
97 this.bestGuess = this.probers.get(0);
98 }
99 }
100
101 return this.bestGuess.getCharSetName();
102 }
103
104 @Override
105 public float getConfidence() {
106 float bestConf = 0.0f;
107 float cf;
108
109 if (this.state == ProbingState.FOUND_IT) {
110 return 0.99f;
111 } else if (this.state == ProbingState.NOT_ME) {
112 return 0.01f;
113 } else {
114 for (CharsetProber prober : this.probers) {
115 if (!prober.isActive()) {
116 continue;
117 }
118 cf = prober.getConfidence();
119 if (bestConf < cf) {
120 bestConf = cf;
121 this.bestGuess = prober;
122 }
123 }
124 }
125
126 return bestConf;
127 }
128
129 @Override
130 public ProbingState getState() {
131 return this.state;
132 }
133
134 @Override
135 public ProbingState handleData(byte[] buf, int offset, int length) {
136 ProbingState st;
137
138 do {
139 ByteBuffer newbuf = filterWithoutEnglishLetters(buf, offset, length);
140 if (newbuf.position() == 0) {
141 break;
142 }
143 for (CharsetProber prober : this.probers) {
144 if (!prober.isActive()) {
145 continue;
146 }
147 st = prober.handleData(newbuf.array(), 0, newbuf.position());
148 if (st == ProbingState.FOUND_IT) {
149 this.bestGuess = prober;
150 this.state = ProbingState.FOUND_IT;
151 break;
152 } else if (st == ProbingState.NOT_ME) {
153 prober.setActive(false);
154 this.activeNum--;
155 if (this.activeNum <= 0) {
156 this.state = ProbingState.NOT_ME;
157 break;
158 }
159 }
160 }
161
162 } while (false);
163
164 return this.state;
165 }
166
167 @Override
168 public final void reset() {
169 this.activeNum = 0;
170 for (CharsetProber prober : this.probers) {
171 prober.reset();
172 prober.setActive(true);
173 this.activeNum++;
174 }
175 this.bestGuess = null;
176 this.state = ProbingState.DETECTING;
177 }
178
179 @Override
180 public void setOption() {
181 }
182 }
+144
-0
src/main/java/org/mozilla/universalchardet/prober/SJISProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
40 import org.mozilla.universalchardet.prober.statemachine.SJISSMModel;
41 import org.mozilla.universalchardet.prober.statemachine.SMModel;
42 import org.mozilla.universalchardet.prober.contextanalysis.SJISContextAnalysis;
43 import org.mozilla.universalchardet.prober.distributionanalysis.SJISDistributionAnalysis;
44 import org.mozilla.universalchardet.Constants;
45
46
47 public class SJISProber extends CharsetProber {
48 ////////////////////////////////////////////////////////////////
49 // fields
50 ////////////////////////////////////////////////////////////////
51 private CodingStateMachine codingSM;
52 private ProbingState state;
53
54 private SJISContextAnalysis contextAnalyzer;
55 private SJISDistributionAnalysis distributionAnalyzer;
56
57 private byte[] lastChar;
58
59 private static final SMModel smModel = new SJISSMModel();
60
61
62 ////////////////////////////////////////////////////////////////
63 // methods
64 ////////////////////////////////////////////////////////////////
65 public SJISProber() {
66 super();
67 this.codingSM = new CodingStateMachine(smModel);
68 this.contextAnalyzer = new SJISContextAnalysis();
69 this.distributionAnalyzer = new SJISDistributionAnalysis();
70 this.lastChar = new byte[2];
71 reset();
72 }
73
74 @Override
75 public String getCharSetName() {
76 return Constants.CHARSET_SHIFT_JIS;
77 }
78
79 @Override
80 public float getConfidence() {
81 float contextCf = this.contextAnalyzer.getConfidence();
82 float distribCf = this.distributionAnalyzer.getConfidence();
83
84 return Math.max(contextCf, distribCf);
85 }
86
87 @Override
88 public ProbingState getState() {
89 return this.state;
90 }
91
92 @Override
93 public ProbingState handleData(byte[] buf, int offset, int length) {
94 int codingState;
95
96 int maxPos = offset + length;
97 for (int i=offset; i<maxPos; ++i) {
98 codingState = this.codingSM.nextState(buf[i]);
99 if (codingState == SMModel.ERROR) {
100 this.state = ProbingState.NOT_ME;
101 break;
102 }
103 if (codingState == SMModel.ITSME) {
104 this.state = ProbingState.FOUND_IT;
105 break;
106 }
107 if (codingState == SMModel.START) {
108 int charLen = this.codingSM.getCurrentCharLen();
109 if (i == offset) {
110 this.lastChar[1] = buf[offset];
111 this.contextAnalyzer.handleOneChar(this.lastChar, 2-charLen, charLen);
112 this.distributionAnalyzer.handleOneChar(this.lastChar, 0, charLen);
113 } else {
114 this.contextAnalyzer.handleOneChar(buf, i+1-charLen, charLen);
115 this.distributionAnalyzer.handleOneChar(buf, i-1, charLen);
116 }
117 }
118 }
119
120 this.lastChar[0] = buf[maxPos-1];
121
122 if (this.state == ProbingState.DETECTING) {
123 if (this.contextAnalyzer.gotEnoughData() && getConfidence() > SHORTCUT_THRESHOLD) {
124 this.state = ProbingState.FOUND_IT;
125 }
126 }
127
128 return this.state;
129 }
130
131 @Override
132 public final void reset() {
133 this.codingSM.reset();
134 this.state = ProbingState.DETECTING;
135 this.contextAnalyzer.reset();
136 this.distributionAnalyzer.reset();
137 java.util.Arrays.fill(this.lastChar, (byte)0);
138 }
139
140 @Override
141 public void setOption()
142 {}
143 }
+186
-0
src/main/java/org/mozilla/universalchardet/prober/SingleByteCharsetProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei TAKETA <k-tak@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import org.mozilla.universalchardet.prober.sequence.SequenceModel;
41
42 public class SingleByteCharsetProber extends CharsetProber {
43 ////////////////////////////////////////////////////////////////
44 // constants
45 ////////////////////////////////////////////////////////////////
46 public static final int SAMPLE_SIZE = 64;
47 public static final int SB_ENOUGH_REL_THRESHOLD = 1024;
48 public static final float POSITIVE_SHORTCUT_THRESHOLD = 0.95f;
49 public static final float NEGATIVE_SHORTCUT_THRESHOLD = 0.05f;
50 public static final int SYMBOL_CAT_ORDER = 250;
51 public static final int NUMBER_OF_SEQ_CAT = 4;
52 public static final int POSITIVE_CAT = NUMBER_OF_SEQ_CAT-1;
53 public static final int NEGATIVE_CAT = 0;
54
55
56 ////////////////////////////////////////////////////////////////
57 // fields
58 ////////////////////////////////////////////////////////////////
59 private ProbingState state;
60 private SequenceModel model;
61 private boolean reversed;
62
63 private short lastOrder;
64
65 private int totalSeqs;
66 private int[] seqCounters;
67
68 private int totalChar;
69 private int freqChar;
70
71 private CharsetProber nameProber;
72
73
74 ////////////////////////////////////////////////////////////////
75 // methods
76 ////////////////////////////////////////////////////////////////
77 public SingleByteCharsetProber(SequenceModel model) {
78 super();
79 this.model = model;
80 this.reversed = false;
81 this.nameProber = null;
82 this.seqCounters = new int[NUMBER_OF_SEQ_CAT];
83 reset();
84 }
85
86 public SingleByteCharsetProber(
87 SequenceModel model,
88 boolean reversed,
89 CharsetProber nameProber)
90 {
91 super();
92 this.model = model;
93 this.reversed = reversed;
94 this.nameProber = nameProber;
95 this.seqCounters = new int[NUMBER_OF_SEQ_CAT];
96 reset();
97 }
98
99 boolean keepEnglishLetters() {
100 return this.model.getKeepEnglishLetter();
101 }
102
103 @Override
104 public String getCharSetName() {
105 if (this.nameProber == null) {
106 return this.model.getCharsetName();
107 } else {
108 return this.nameProber.getCharSetName();
109 }
110 }
111
112 @Override
113 public float getConfidence() {
114 if (this.totalSeqs > 0) {
115 float r = 1.0f * this.seqCounters[POSITIVE_CAT] / this.totalSeqs / this.model.getTypicalPositiveRatio();
116 r = r * this.freqChar / this.totalChar;
117 if (r >= 1.0f) {
118 r = 0.99f;
119 }
120 return r;
121 }
122
123 return 0.01f;
124 }
125
126 @Override
127 public ProbingState getState() {
128 return this.state;
129 }
130
131 @Override
132 public ProbingState handleData(byte[] buf, int offset, int length) {
133 short order;
134
135 int maxPos = offset + length;
136 for (int i=offset; i<maxPos; ++i) {
137 order = this.model.getOrder(buf[i]);
138
139 if (order < SYMBOL_CAT_ORDER) {
140 ++this.totalChar;
141 }
142 if (order < SAMPLE_SIZE) {
143 ++this.freqChar;
144 if (this.lastOrder < SAMPLE_SIZE) {
145 ++this.totalSeqs;
146 if (!this.reversed) {
147 ++(this.seqCounters[this.model.getPrecedence(this.lastOrder*SAMPLE_SIZE+order)]);
148 } else {
149 ++(this.seqCounters[this.model.getPrecedence(order*SAMPLE_SIZE+this.lastOrder)]);
150 }
151 }
152 }
153 this.lastOrder = order;
154 }
155
156 if (this.state == ProbingState.DETECTING) {
157 if (this.totalSeqs > SB_ENOUGH_REL_THRESHOLD) {
158 float cf = getConfidence();
159 if (cf > POSITIVE_SHORTCUT_THRESHOLD) {
160 this.state = ProbingState.FOUND_IT;
161 } else if (cf < NEGATIVE_SHORTCUT_THRESHOLD){
162 this.state = ProbingState.NOT_ME;
163 }
164 }
165 }
166
167 return this.state;
168 }
169
170 @Override
171 public final void reset() {
172 this.state = ProbingState.DETECTING;
173 this.lastOrder = 255;
174 for (int i=0; i<NUMBER_OF_SEQ_CAT; ++i) {
175 this.seqCounters[i] = 0;
176 }
177 this.totalSeqs = 0;
178 this.totalChar = 0;
179 this.freqChar = 0;
180 }
181
182 @Override
183 public void setOption()
184 {}
185 }
+133
-0
src/main/java/org/mozilla/universalchardet/prober/UTF8Prober.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
40 import org.mozilla.universalchardet.prober.statemachine.SMModel;
41 import org.mozilla.universalchardet.prober.statemachine.UTF8SMModel;
42 import org.mozilla.universalchardet.Constants;
43
44
45 public class UTF8Prober extends CharsetProber {
46 ////////////////////////////////////////////////////////////////
47 // constants
48 ////////////////////////////////////////////////////////////////
49 public static final float ONE_CHAR_PROB = 0.50f;
50
51
52 ////////////////////////////////////////////////////////////////
53 // fields
54 ////////////////////////////////////////////////////////////////
55 private CodingStateMachine codingSM;
56 private ProbingState state;
57 private int numOfMBChar;
58
59 private static final SMModel smModel = new UTF8SMModel();
60
61
62 ////////////////////////////////////////////////////////////////
63 // methods
64 ////////////////////////////////////////////////////////////////
65 public UTF8Prober() {
66 super();
67 this.numOfMBChar = 0;
68 this.codingSM = new CodingStateMachine(smModel);
69
70 reset();
71 }
72
73 public String getCharSetName() {
74 return Constants.CHARSET_UTF_8;
75 }
76
77 public ProbingState handleData(final byte[] buf, int offset, int length) {
78 int codingState;
79
80 int maxPos = offset + length;
81 for (int i=offset; i<maxPos; ++i) {
82 codingState = this.codingSM.nextState(buf[i]);
83 if (codingState == SMModel.ERROR) {
84 this.state = ProbingState.NOT_ME;
85 break;
86 }
87 if (codingState == SMModel.ITSME) {
88 this.state = ProbingState.FOUND_IT;
89 break;
90 }
91 if (codingState == SMModel.START) {
92 if (this.codingSM.getCurrentCharLen() >= 2) {
93 ++this.numOfMBChar;
94 }
95 }
96 }
97
98 if (this.state == ProbingState.DETECTING) {
99 if (getConfidence() > SHORTCUT_THRESHOLD) {
100 this.state = ProbingState.FOUND_IT;
101 }
102 }
103
104 return this.state;
105 }
106
107 public ProbingState getState() {
108 return this.state;
109 }
110
111 public final void reset() {
112 this.codingSM.reset();
113 this.numOfMBChar = 0;
114 this.state = ProbingState.DETECTING;
115 }
116
117 public float getConfidence() {
118 float unlike = 0.99f;
119
120 if (this.numOfMBChar < 6) {
121 for (int i=0; i<this.numOfMBChar; ++i) {
122 unlike *= ONE_CHAR_PROB;
123 }
124 return (1.0f - unlike);
125 } else {
126 return 0.99f;
127 }
128 }
129
130 public void setOption()
131 {}
132 }
+97
-0
src/main/java/org/mozilla/universalchardet/prober/contextanalysis/EUCJPContextAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.contextanalysis;
38
39
40 public class EUCJPContextAnalysis extends JapaneseContextAnalysis {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final int HIRAGANA_HIGHBYTE = 0xA4;
45 public static final int HIRAGANA_LOWBYTE_BEGIN = 0xA1;
46 public static final int HIRAGANA_LOWBYTE_END = 0xF3;
47 public static final int SINGLE_SHIFT_2 = 0x8E;
48 public static final int SINGLE_SHIFT_3 = 0x8F;
49 public static final int FIRSTPLANE_HIGHBYTE_BEGIN = 0xA1;
50 public static final int FIRSTPLANE_HIGHBYTE_END = 0xFE;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public EUCJPContextAnalysis() {
57 super();
58 }
59
60 @Override
61 protected void getOrder(Order order, final byte[] buf, int offset) {
62 order.order = -1;
63 order.charLength = 1;
64
65 int firstByte = buf[offset] & 0xFF;
66 if (firstByte == SINGLE_SHIFT_2 ||
67 (firstByte >= FIRSTPLANE_HIGHBYTE_BEGIN &&
68 firstByte <= FIRSTPLANE_HIGHBYTE_END)) {
69 order.charLength = 2;
70 } else if (firstByte == SINGLE_SHIFT_3) {
71 order.charLength = 3;
72 }
73
74 if (firstByte == HIRAGANA_HIGHBYTE) {
75 int secondByte = buf[offset+1] & 0xFF;
76 if (secondByte >= HIRAGANA_LOWBYTE_BEGIN &&
77 secondByte <= HIRAGANA_LOWBYTE_END) {
78 order.order = (secondByte - HIRAGANA_LOWBYTE_BEGIN);
79 }
80 }
81 }
82
83 @Override
84 protected int getOrder(final byte[] buf, int offset) {
85 int highbyte = buf[offset] & 0xFF;
86 if (highbyte == HIRAGANA_HIGHBYTE) {
87 int lowbyte = buf[offset+1] & 0xFF;
88 if (lowbyte >= HIRAGANA_LOWBYTE_BEGIN &&
89 lowbyte <= HIRAGANA_LOWBYTE_END) {
90 return (lowbyte - HIRAGANA_LOWBYTE_BEGIN);
91 }
92 }
93
94 return -1;
95 }
96 }
+255
-0
src/main/java/org/mozilla/universalchardet/prober/contextanalysis/JapaneseContextAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.contextanalysis;
38
39 public abstract class JapaneseContextAnalysis {
40 ////////////////////////////////////////////////////////////////
41 // constants
42 ////////////////////////////////////////////////////////////////
43 public static final int NUM_OF_CATEGORY = 6;
44 public static final int ENOUGH_REL_THRESHOLD = 100;
45 public static final int MAX_REL_THRESHOLD = 1000;
46 public static final int MINIMUM_DATA_THRESHOLD = 4;
47 public static final float DONT_KNOW = -1f;
48
49
50 ////////////////////////////////////////////////////////////////
51 // inner types
52 ////////////////////////////////////////////////////////////////
53 protected static class Order {
54 public int order;
55 public int charLength;
56
57 public Order()
58 {
59 this.order = -1;
60 this.charLength = 0;
61 }
62 }
63
64 ////////////////////////////////////////////////////////////////
65 // fields
66 ////////////////////////////////////////////////////////////////
67 private int[] relSample = new int[NUM_OF_CATEGORY];
68 private int totalRel;
69 private int lastCharOrder;
70 private int needToSkipCharNum;
71 private boolean done;
72
73 private Order tmpOrder;
74
75
76 ////////////////////////////////////////////////////////////////
77 // methods
78 ////////////////////////////////////////////////////////////////
79 public JapaneseContextAnalysis() {
80 super();
81 tmpOrder = new Order();
82 reset();
83 }
84
85 public void handleData(final byte[] buf, int offset, int length) {
86 if (this.done) {
87 return;
88 }
89
90 // The buffer we got is byte oriented, and a character may span in more than one
91 // buffers. In case the last one or two byte in last buffer is not complete, we
92 // record how many byte needed to complete that character and skip these bytes here.
93 // We can choose to record those bytes as well and analyse the character once it
94 // is complete, but since a character will not make much difference, by simply skipping
95 // this character will simply our logic and improve performance.
96 int maxPos = offset + length;
97
98 for (int i=this.needToSkipCharNum+offset; i<maxPos; ) {
99 getOrder(this.tmpOrder, buf, i);
100 i += this.tmpOrder.charLength;
101
102 if (i > maxPos) {
103 this.needToSkipCharNum = i - maxPos;
104 this.lastCharOrder = -1;
105 } else {
106 if (this.tmpOrder.order != -1 && this.lastCharOrder != -1) {
107 ++this.totalRel;
108 if (this.totalRel > MAX_REL_THRESHOLD) {
109 this.done = true;
110 break;
111 }
112 ++this.relSample[jp2CharContext[this.lastCharOrder][this.tmpOrder.order]];
113 }
114 }
115 }
116 }
117
118 public void handleOneChar(final byte[] buf, int offset, int charLength) {
119 if (this.totalRel > MAX_REL_THRESHOLD) {
120 this.done = true;
121 }
122 if (this.done) {
123 return;
124 }
125
126
127 int orderNum = -1;
128 if (charLength == 2) {
129 orderNum = getOrder(buf, offset);
130 }
131 if (orderNum != -1 && this.lastCharOrder != -1) {
132 ++this.totalRel;
133 ++this.relSample[jp2CharContext[this.lastCharOrder][orderNum]];
134 }
135 this.lastCharOrder = orderNum;
136 }
137
138 public float getConfidence() {
139 if (this.totalRel > MINIMUM_DATA_THRESHOLD) {
140 return ((float)(this.totalRel - this.relSample[0])) / this.totalRel;
141 } else {
142 return DONT_KNOW;
143 }
144 }
145
146 public final void reset() {
147 this.totalRel = 0;
148 for (int i=0; i<NUM_OF_CATEGORY; ++i) {
149 this.relSample[i] = 0;
150 }
151 this.needToSkipCharNum = 0;
152 this.lastCharOrder = -1;
153 this.done = false;
154 }
155
156 public void setOption()
157 {}
158
159 public boolean gotEnoughData() {
160 return (this.totalRel > ENOUGH_REL_THRESHOLD);
161 }
162
163 protected abstract void getOrder(Order order, final byte[] buf, int offset);
164 protected abstract int getOrder(final byte[] buf, int offset);
165
166 ////////////////////////////////////////////////////////////////
167 // constants continued
168 ////////////////////////////////////////////////////////////////
169 private static final byte[][] jp2CharContext = new byte[][] {
170 { 0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,},
171 { 2,4,0,4,0,3,0,4,0,3,4,4,4,2,4,3,3,4,3,2,3,3,4,2,3,3,3,2,4,1,4,3,3,1,5,4,3,4,3,4,3,5,3,0,3,5,4,2,0,3,1,0,3,3,0,3,3,0,1,1,0,4,3,0,3,3,0,4,0,2,0,3,5,5,5,5,4,0,4,1,0,3,4,},
172 { 0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,},
173 { 0,4,0,5,0,5,0,4,0,4,5,4,4,3,5,3,5,1,5,3,4,3,4,4,3,4,3,3,4,3,5,4,4,3,5,5,3,5,5,5,3,5,5,3,4,5,5,3,1,3,2,0,3,4,0,4,2,0,4,2,1,5,3,2,3,5,0,4,0,2,0,5,4,4,5,4,5,0,4,0,0,4,4,},
174 { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,},
175 { 0,3,0,4,0,3,0,3,0,4,5,4,3,3,3,3,4,3,5,4,4,3,5,4,4,3,4,3,4,4,4,4,5,3,4,4,3,4,5,5,4,5,5,1,4,5,4,3,0,3,3,1,3,3,0,4,4,0,3,3,1,5,3,3,3,5,0,4,0,3,0,4,4,3,4,3,3,0,4,1,1,3,4,},
176 { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,},
177 { 0,4,0,3,0,3,0,4,0,3,4,4,3,2,2,1,2,1,3,1,3,3,3,3,3,4,3,1,3,3,5,3,3,0,4,3,0,5,4,3,3,5,4,4,3,4,4,5,0,1,2,0,1,2,0,2,2,0,1,0,0,5,2,2,1,4,0,3,0,1,0,4,4,3,5,4,3,0,2,1,0,4,3,},
178 { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,},
179 { 0,3,0,5,0,4,0,2,1,4,4,2,4,1,4,2,4,2,4,3,3,3,4,3,3,3,3,1,4,2,3,3,3,1,4,4,1,1,1,4,3,3,2,0,2,4,3,2,0,3,3,0,3,1,1,0,0,0,3,3,0,4,2,2,3,4,0,4,0,3,0,4,4,5,3,4,4,0,3,0,0,1,4,},
180 { 1,4,0,4,0,4,0,4,0,3,5,4,4,3,4,3,5,4,3,3,4,3,5,4,4,4,4,3,4,2,4,3,3,1,5,4,3,2,4,5,4,5,5,4,4,5,4,4,0,3,2,2,3,3,0,4,3,1,3,2,1,4,3,3,4,5,0,3,0,2,0,4,5,5,4,5,4,0,4,0,0,5,4,},
181 { 0,5,0,5,0,4,0,3,0,4,4,3,4,3,3,3,4,0,4,4,4,3,4,3,4,3,3,1,4,2,4,3,4,0,5,4,1,4,5,4,4,5,3,2,4,3,4,3,2,4,1,3,3,3,2,3,2,0,4,3,3,4,3,3,3,4,0,4,0,3,0,4,5,4,4,4,3,0,4,1,0,1,3,},
182 { 0,3,1,4,0,3,0,2,0,3,4,4,3,1,4,2,3,3,4,3,4,3,4,3,4,4,3,2,3,1,5,4,4,1,4,4,3,5,4,4,3,5,5,4,3,4,4,3,1,2,3,1,2,2,0,3,2,0,3,1,0,5,3,3,3,4,3,3,3,3,4,4,4,4,5,4,2,0,3,3,2,4,3,},
183 { 0,2,0,3,0,1,0,1,0,0,3,2,0,0,2,0,1,0,2,1,3,3,3,1,2,3,1,0,1,0,4,2,1,1,3,3,0,4,3,3,1,4,3,3,0,3,3,2,0,0,0,0,1,0,0,2,0,0,0,0,0,4,1,0,2,3,2,2,2,1,3,3,3,4,4,3,2,0,3,1,0,3,3,},
184 { 0,4,0,4,0,3,0,3,0,4,4,4,3,3,3,3,3,3,4,3,4,2,4,3,4,3,3,2,4,3,4,5,4,1,4,5,3,5,4,5,3,5,4,0,3,5,5,3,1,3,3,2,2,3,0,3,4,1,3,3,2,4,3,3,3,4,0,4,0,3,0,4,5,4,4,5,3,0,4,1,0,3,4,},
185 { 0,2,0,3,0,3,0,0,0,2,2,2,1,0,1,0,0,0,3,0,3,0,3,0,1,3,1,0,3,1,3,3,3,1,3,3,3,0,1,3,1,3,4,0,0,3,1,1,0,3,2,0,0,0,0,1,3,0,1,0,0,3,3,2,0,3,0,0,0,0,0,3,4,3,4,3,3,0,3,0,0,2,3,},
186 { 2,3,0,3,0,2,0,1,0,3,3,4,3,1,3,1,1,1,3,1,4,3,4,3,3,3,0,0,3,1,5,4,3,1,4,3,2,5,5,4,4,4,4,3,3,4,4,4,0,2,1,1,3,2,0,1,2,0,0,1,0,4,1,3,3,3,0,3,0,1,0,4,4,4,5,5,3,0,2,0,0,4,4,},
187 { 0,2,0,1,0,3,1,3,0,2,3,3,3,0,3,1,0,0,3,0,3,2,3,1,3,2,1,1,0,0,4,2,1,0,2,3,1,4,3,2,0,4,4,3,1,3,1,3,0,1,0,0,1,0,0,0,1,0,0,0,0,4,1,1,1,2,0,3,0,0,0,3,4,2,4,3,2,0,1,0,0,3,3,},
188 { 0,1,0,4,0,5,0,4,0,2,4,4,2,3,3,2,3,3,5,3,3,3,4,3,4,2,3,0,4,3,3,3,4,1,4,3,2,1,5,5,3,4,5,1,3,5,4,2,0,3,3,0,1,3,0,4,2,0,1,3,1,4,3,3,3,3,0,3,0,1,0,3,4,4,4,5,5,0,3,0,1,4,5,},
189 { 0,2,0,3,0,3,0,0,0,2,3,1,3,0,4,0,1,1,3,0,3,4,3,2,3,1,0,3,3,2,3,1,3,0,2,3,0,2,1,4,1,2,2,0,0,3,3,0,0,2,0,0,0,1,0,0,0,0,2,2,0,3,2,1,3,3,0,2,0,2,0,0,3,3,1,2,4,0,3,0,2,2,3,},
190 { 2,4,0,5,0,4,0,4,0,2,4,4,4,3,4,3,3,3,1,2,4,3,4,3,4,4,5,0,3,3,3,3,2,0,4,3,1,4,3,4,1,4,4,3,3,4,4,3,1,2,3,0,4,2,0,4,1,0,3,3,0,4,3,3,3,4,0,4,0,2,0,3,5,3,4,5,2,0,3,0,0,4,5,},
191 { 0,3,0,4,0,1,0,1,0,1,3,2,2,1,3,0,3,0,2,0,2,0,3,0,2,0,0,0,1,0,1,1,0,0,3,1,0,0,0,4,0,3,1,0,2,1,3,0,0,0,0,0,0,3,0,0,0,0,0,0,0,4,2,2,3,1,0,3,0,0,0,1,4,4,4,3,0,0,4,0,0,1,4,},
192 { 1,4,1,5,0,3,0,3,0,4,5,4,4,3,5,3,3,4,4,3,4,1,3,3,3,3,2,1,4,1,5,4,3,1,4,4,3,5,4,4,3,5,4,3,3,4,4,4,0,3,3,1,2,3,0,3,1,0,3,3,0,5,4,4,4,4,4,4,3,3,5,4,4,3,3,5,4,0,3,2,0,4,4,},
193 { 0,2,0,3,0,1,0,0,0,1,3,3,3,2,4,1,3,0,3,1,3,0,2,2,1,1,0,0,2,0,4,3,1,0,4,3,0,4,4,4,1,4,3,1,1,3,3,1,0,2,0,0,1,3,0,0,0,0,2,0,0,4,3,2,4,3,5,4,3,3,3,4,3,3,4,3,3,0,2,1,0,3,3,},
194 { 0,2,0,4,0,3,0,2,0,2,5,5,3,4,4,4,4,1,4,3,3,0,4,3,4,3,1,3,3,2,4,3,0,3,4,3,0,3,4,4,2,4,4,0,4,5,3,3,2,2,1,1,1,2,0,1,5,0,3,3,2,4,3,3,3,4,0,3,0,2,0,4,4,3,5,5,0,0,3,0,2,3,3,},
195 { 0,3,0,4,0,3,0,1,0,3,4,3,3,1,3,3,3,0,3,1,3,0,4,3,3,1,1,0,3,0,3,3,0,0,4,4,0,1,5,4,3,3,5,0,3,3,4,3,0,2,0,1,1,1,0,1,3,0,1,2,1,3,3,2,3,3,0,3,0,1,0,1,3,3,4,4,1,0,1,2,2,1,3,},
196 { 0,1,0,4,0,4,0,3,0,1,3,3,3,2,3,1,1,0,3,0,3,3,4,3,2,4,2,0,1,0,4,3,2,0,4,3,0,5,3,3,2,4,4,4,3,3,3,4,0,1,3,0,0,1,0,0,1,0,0,0,0,4,2,3,3,3,0,3,0,0,0,4,4,4,5,3,2,0,3,3,0,3,5,},
197 { 0,2,0,3,0,0,0,3,0,1,3,0,2,0,0,0,1,0,3,1,1,3,3,0,0,3,0,0,3,0,2,3,1,0,3,1,0,3,3,2,0,4,2,2,0,2,0,0,0,4,0,0,0,0,0,0,0,0,0,0,0,2,1,2,0,1,0,1,0,0,0,1,3,1,2,0,0,0,1,0,0,1,4,},
198 { 0,3,0,3,0,5,0,1,0,2,4,3,1,3,3,2,1,1,5,2,1,0,5,1,2,0,0,0,3,3,2,2,3,2,4,3,0,0,3,3,1,3,3,0,2,5,3,4,0,3,3,0,1,2,0,2,2,0,3,2,0,2,2,3,3,3,0,2,0,1,0,3,4,4,2,5,4,0,3,0,0,3,5,},
199 { 0,3,0,3,0,3,0,1,0,3,3,3,3,0,3,0,2,0,2,1,1,0,2,0,1,0,0,0,2,1,0,0,1,0,3,2,0,0,3,3,1,2,3,1,0,3,3,0,0,1,0,0,0,0,0,2,0,0,0,0,0,2,3,1,2,3,0,3,0,1,0,3,2,1,0,4,3,0,1,1,0,3,3,},
200 { 0,4,0,5,0,3,0,3,0,4,5,5,4,3,5,3,4,3,5,3,3,2,5,3,4,4,4,3,4,3,4,5,5,3,4,4,3,4,4,5,4,4,4,3,4,5,5,4,2,3,4,2,3,4,0,3,3,1,4,3,2,4,3,3,5,5,0,3,0,3,0,5,5,5,5,4,4,0,4,0,1,4,4,},
201 { 0,4,0,4,0,3,0,3,0,3,5,4,4,2,3,2,5,1,3,2,5,1,4,2,3,2,3,3,4,3,3,3,3,2,5,4,1,3,3,5,3,4,4,0,4,4,3,1,1,3,1,0,2,3,0,2,3,0,3,0,0,4,3,1,3,4,0,3,0,2,0,4,4,4,3,4,5,0,4,0,0,3,4,},
202 { 0,3,0,3,0,3,1,2,0,3,4,4,3,3,3,0,2,2,4,3,3,1,3,3,3,1,1,0,3,1,4,3,2,3,4,4,2,4,4,4,3,4,4,3,2,4,4,3,1,3,3,1,3,3,0,4,1,0,2,2,1,4,3,2,3,3,5,4,3,3,5,4,4,3,3,0,4,0,3,2,2,4,4,},
203 { 0,2,0,1,0,0,0,0,0,1,2,1,3,0,0,0,0,0,2,0,1,2,1,0,0,1,0,0,0,0,3,0,0,1,0,1,1,3,1,0,0,0,1,1,0,1,1,0,0,0,0,0,2,0,0,0,0,0,0,0,0,1,1,2,2,0,3,4,0,0,0,1,1,0,0,1,0,0,0,0,0,1,1,},
204 { 0,1,0,0,0,1,0,0,0,0,4,0,4,1,4,0,3,0,4,0,3,0,4,0,3,0,3,0,4,1,5,1,4,0,0,3,0,5,0,5,2,0,1,0,0,0,2,1,4,0,1,3,0,0,3,0,0,3,1,1,4,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,},
205 { 1,4,0,5,0,3,0,2,0,3,5,4,4,3,4,3,5,3,4,3,3,0,4,3,3,3,3,3,3,2,4,4,3,1,3,4,4,5,4,4,3,4,4,1,3,5,4,3,3,3,1,2,2,3,3,1,3,1,3,3,3,5,3,3,4,5,0,3,0,3,0,3,4,3,4,4,3,0,3,0,2,4,3,},
206 { 0,1,0,4,0,0,0,0,0,1,4,0,4,1,4,2,4,0,3,0,1,0,1,0,0,0,0,0,2,0,3,1,1,1,0,3,0,0,0,1,2,1,0,0,1,1,1,1,0,1,0,0,0,1,0,0,3,0,0,0,0,3,2,0,2,2,0,1,0,0,0,2,3,2,3,3,0,0,0,0,2,1,0,},
207 { 0,5,1,5,0,3,0,3,0,5,4,4,5,1,5,3,3,0,4,3,4,3,5,3,4,3,3,2,4,3,4,3,3,0,3,3,1,4,4,3,4,4,4,3,4,5,5,3,2,3,1,1,3,3,1,3,1,1,3,3,2,4,5,3,3,5,0,4,0,3,0,4,4,3,5,3,3,0,3,4,0,4,3,},
208 { 0,5,0,5,0,3,0,2,0,4,4,3,5,2,4,3,3,3,4,4,4,3,5,3,5,3,3,1,4,0,4,3,3,0,3,3,0,4,4,4,4,5,4,3,3,5,5,3,2,3,1,2,3,2,0,1,0,0,3,2,2,4,4,3,1,5,0,4,0,3,0,4,3,1,3,2,1,0,3,3,0,3,3,},
209 { 0,4,0,5,0,5,0,4,0,4,5,5,5,3,4,3,3,2,5,4,4,3,5,3,5,3,4,0,4,3,4,4,3,2,4,4,3,4,5,4,4,5,5,0,3,5,5,4,1,3,3,2,3,3,1,3,1,0,4,3,1,4,4,3,4,5,0,4,0,2,0,4,3,4,4,3,3,0,4,0,0,5,5,},
210 { 0,4,0,4,0,5,0,1,1,3,3,4,4,3,4,1,3,0,5,1,3,0,3,1,3,1,1,0,3,0,3,3,4,0,4,3,0,4,4,4,3,4,4,0,3,5,4,1,0,3,0,0,2,3,0,3,1,0,3,1,0,3,2,1,3,5,0,3,0,1,0,3,2,3,3,4,4,0,2,2,0,4,4,},
211 { 2,4,0,5,0,4,0,3,0,4,5,5,4,3,5,3,5,3,5,3,5,2,5,3,4,3,3,4,3,4,5,3,2,1,5,4,3,2,3,4,5,3,4,1,2,5,4,3,0,3,3,0,3,2,0,2,3,0,4,1,0,3,4,3,3,5,0,3,0,1,0,4,5,5,5,4,3,0,4,2,0,3,5,},
212 { 0,5,0,4,0,4,0,2,0,5,4,3,4,3,4,3,3,3,4,3,4,2,5,3,5,3,4,1,4,3,4,4,4,0,3,5,0,4,4,4,4,5,3,1,3,4,5,3,3,3,3,3,3,3,0,2,2,0,3,3,2,4,3,3,3,5,3,4,1,3,3,5,3,2,0,0,0,0,4,3,1,3,3,},
213 { 0,1,0,3,0,3,0,1,0,1,3,3,3,2,3,3,3,0,3,0,0,0,3,1,3,0,0,0,2,2,2,3,0,0,3,2,0,1,2,4,1,3,3,0,0,3,3,3,0,1,0,0,2,1,0,0,3,0,3,1,0,3,0,0,1,3,0,2,0,1,0,3,3,1,3,3,0,0,1,1,0,3,3,},
214 { 0,2,0,3,0,2,1,4,0,2,2,3,1,1,3,1,1,0,2,0,3,1,2,3,1,3,0,0,1,0,4,3,2,3,3,3,1,4,2,3,3,3,3,1,0,3,1,4,0,1,1,0,1,2,0,1,1,0,1,1,0,3,1,3,2,2,0,1,0,0,0,2,3,3,3,1,0,0,0,0,0,2,3,},
215 { 0,5,0,4,0,5,0,2,0,4,5,5,3,3,4,3,3,1,5,4,4,2,4,4,4,3,4,2,4,3,5,5,4,3,3,4,3,3,5,5,4,5,5,1,3,4,5,3,1,4,3,1,3,3,0,3,3,1,4,3,1,4,5,3,3,5,0,4,0,3,0,5,3,3,1,4,3,0,4,0,1,5,3,},
216 { 0,5,0,5,0,4,0,2,0,4,4,3,4,3,3,3,3,3,5,4,4,4,4,4,4,5,3,3,5,2,4,4,4,3,4,4,3,3,4,4,5,5,3,3,4,3,4,3,3,4,3,3,3,3,1,2,2,1,4,3,3,5,4,4,3,4,0,4,0,3,0,4,4,4,4,4,1,0,4,2,0,2,4,},
217 { 0,4,0,4,0,3,0,1,0,3,5,2,3,0,3,0,2,1,4,2,3,3,4,1,4,3,3,2,4,1,3,3,3,0,3,3,0,0,3,3,3,5,3,3,3,3,3,2,0,2,0,0,2,0,0,2,0,0,1,0,0,3,1,2,2,3,0,3,0,2,0,4,4,3,3,4,1,0,3,0,0,2,4,},
218 { 0,0,0,4,0,0,0,0,0,0,1,0,1,0,2,0,0,0,0,0,1,0,2,0,1,0,0,0,0,0,3,1,3,0,3,2,0,0,0,1,0,3,2,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,4,0,2,0,0,0,0,0,0,2,},
219 { 0,2,1,3,0,2,0,2,0,3,3,3,3,1,3,1,3,3,3,3,3,3,4,2,2,1,2,1,4,0,4,3,1,3,3,3,2,4,3,5,4,3,3,3,3,3,3,3,0,1,3,0,2,0,0,1,0,0,1,0,0,4,2,0,2,3,0,3,3,0,3,3,4,2,3,1,4,0,1,2,0,2,3,},
220 { 0,3,0,3,0,1,0,3,0,2,3,3,3,0,3,1,2,0,3,3,2,3,3,2,3,2,3,1,3,0,4,3,2,0,3,3,1,4,3,3,2,3,4,3,1,3,3,1,1,0,1,1,0,1,0,1,0,1,0,0,0,4,1,1,0,3,0,3,1,0,2,3,3,3,3,3,1,0,0,2,0,3,3,},
221 { 0,0,0,0,0,0,0,0,0,0,3,0,2,0,3,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,3,0,3,0,3,1,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,2,0,2,3,0,0,0,0,0,0,0,0,3,},
222 { 0,2,0,3,1,3,0,3,0,2,3,3,3,1,3,1,3,1,3,1,3,3,3,1,3,0,2,3,1,1,4,3,3,2,3,3,1,2,2,4,1,3,3,0,1,4,2,3,0,1,3,0,3,0,0,1,3,0,2,0,0,3,3,2,1,3,0,3,0,2,0,3,4,4,4,3,1,0,3,0,0,3,3,},
223 { 0,2,0,1,0,2,0,0,0,1,3,2,2,1,3,0,1,1,3,0,3,2,3,1,2,0,2,0,1,1,3,3,3,0,3,3,1,1,2,3,2,3,3,1,2,3,2,0,0,1,0,0,0,0,0,0,3,0,1,0,0,2,1,2,1,3,0,3,0,0,0,3,4,4,4,3,2,0,2,0,0,2,4,},
224 { 0,0,0,1,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,1,1,1,0,0,0,0,0,0,0,0,0,2,2,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,3,1,0,0,0,0,0,0,0,3,},
225 { 0,3,0,3,0,2,0,3,0,3,3,3,2,3,2,2,2,0,3,1,3,3,3,2,3,3,0,0,3,0,3,2,2,0,2,3,1,4,3,4,3,3,2,3,1,5,4,4,0,3,1,2,1,3,0,3,1,1,2,0,2,3,1,3,1,3,0,3,0,1,0,3,3,4,4,2,1,0,2,1,0,2,4,},
226 { 0,1,0,3,0,1,0,2,0,1,4,2,5,1,4,0,2,0,2,1,3,1,4,0,2,1,0,0,2,1,4,1,1,0,3,3,0,5,1,3,2,3,3,1,0,3,2,3,0,1,0,0,0,0,0,0,1,0,0,0,0,4,0,1,0,3,0,2,0,1,0,3,3,3,4,3,3,0,0,0,0,2,3,},
227 { 0,0,0,1,0,0,0,0,0,0,2,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,1,0,0,1,0,0,0,0,0,3,},
228 { 0,1,0,3,0,4,0,3,0,2,4,3,1,0,3,2,2,1,3,1,2,2,3,1,1,1,2,1,3,0,1,2,0,1,3,2,1,3,0,5,5,1,0,0,1,3,2,1,0,3,0,0,1,0,0,0,0,0,3,4,0,1,1,1,3,2,0,2,0,1,0,2,3,3,1,2,3,0,1,0,1,0,4,},
229 { 0,0,0,1,0,3,0,3,0,2,2,1,0,0,4,0,3,0,3,1,3,0,3,0,3,0,1,0,3,0,3,1,3,0,3,3,0,0,1,2,1,1,1,0,1,2,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,2,2,1,2,0,0,2,0,0,0,0,2,3,3,3,3,0,0,0,0,1,4,},
230 { 0,0,0,3,0,3,0,0,0,0,3,1,1,0,3,0,1,0,2,0,1,0,0,0,0,0,0,0,1,0,3,0,2,0,2,3,0,0,2,2,3,1,2,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,2,0,0,0,0,2,3,},
231 { 2,4,0,5,0,5,0,4,0,3,4,3,3,3,4,3,3,3,4,3,4,4,5,4,5,5,5,2,3,0,5,5,4,1,5,4,3,1,5,4,3,4,4,3,3,4,3,3,0,3,2,0,2,3,0,3,0,0,3,3,0,5,3,2,3,3,0,3,0,3,0,3,4,5,4,5,3,0,4,3,0,3,4,},
232 { 0,3,0,3,0,3,0,3,0,3,3,4,3,2,3,2,3,0,4,3,3,3,3,3,3,3,3,0,3,2,4,3,3,1,3,4,3,4,4,4,3,4,4,3,2,4,4,1,0,2,0,0,1,1,0,2,0,0,3,1,0,5,3,2,1,3,0,3,0,1,2,4,3,2,4,3,3,0,3,2,0,4,4,},
233 { 0,3,0,3,0,1,0,0,0,1,4,3,3,2,3,1,3,1,4,2,3,2,4,2,3,4,3,0,2,2,3,3,3,0,3,3,3,0,3,4,1,3,3,0,3,4,3,3,0,1,1,0,1,0,0,0,4,0,3,0,0,3,1,2,1,3,0,4,0,1,0,4,3,3,4,3,3,0,2,0,0,3,3,},
234 { 0,3,0,4,0,1,0,3,0,3,4,3,3,0,3,3,3,1,3,1,3,3,4,3,3,3,0,0,3,1,5,3,3,1,3,3,2,5,4,3,3,4,5,3,2,5,3,4,0,1,0,0,0,0,0,2,0,0,1,1,0,4,2,2,1,3,0,3,0,2,0,4,4,3,5,3,2,0,1,1,0,3,4,},
235 { 0,5,0,4,0,5,0,2,0,4,4,3,3,2,3,3,3,1,4,3,4,1,5,3,4,3,4,0,4,2,4,3,4,1,5,4,0,4,4,4,4,5,4,1,3,5,4,2,1,4,1,1,3,2,0,3,1,0,3,2,1,4,3,3,3,4,0,4,0,3,0,4,4,4,3,3,3,0,4,2,0,3,4,},
236 { 1,4,0,4,0,3,0,1,0,3,3,3,1,1,3,3,2,2,3,3,1,0,3,2,2,1,2,0,3,1,2,1,2,0,3,2,0,2,2,3,3,4,3,0,3,3,1,2,0,1,1,3,1,2,0,0,3,0,1,1,0,3,2,2,3,3,0,3,0,0,0,2,3,3,4,3,3,0,1,0,0,1,4,},
237 { 0,4,0,4,0,4,0,0,0,3,4,4,3,1,4,2,3,2,3,3,3,1,4,3,4,0,3,0,4,2,3,3,2,2,5,4,2,1,3,4,3,4,3,1,3,3,4,2,0,2,1,0,3,3,0,0,2,0,3,1,0,4,4,3,4,3,0,4,0,1,0,2,4,4,4,4,4,0,3,2,0,3,3,},
238 { 0,0,0,1,0,4,0,0,0,0,0,0,1,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,3,2,0,0,1,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,2,},
239 { 0,2,0,3,0,4,0,4,0,1,3,3,3,0,4,0,2,1,2,1,1,1,2,0,3,1,1,0,1,0,3,1,0,0,3,3,2,0,1,1,0,0,0,0,0,1,0,2,0,2,2,0,3,1,0,0,1,0,1,1,0,1,2,0,3,0,0,0,0,1,0,0,3,3,4,3,1,0,1,0,3,0,2,},
240 { 0,0,0,3,0,5,0,0,0,0,1,0,2,0,3,1,0,1,3,0,0,0,2,0,0,0,1,0,0,0,1,1,0,0,4,0,0,0,2,3,0,1,4,1,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,1,0,0,0,0,0,0,0,2,0,0,3,0,0,0,0,0,3,},
241 { 0,2,0,5,0,5,0,1,0,2,4,3,3,2,5,1,3,2,3,3,3,0,4,1,2,0,3,0,4,0,2,2,1,1,5,3,0,0,1,4,2,3,2,0,3,3,3,2,0,2,4,1,1,2,0,1,1,0,3,1,0,1,3,1,2,3,0,2,0,0,0,1,3,5,4,4,4,0,3,0,0,1,3,},
242 { 0,4,0,5,0,4,0,4,0,4,5,4,3,3,4,3,3,3,4,3,4,4,5,3,4,5,4,2,4,2,3,4,3,1,4,4,1,3,5,4,4,5,5,4,4,5,5,5,2,3,3,1,4,3,1,3,3,0,3,3,1,4,3,4,4,4,0,3,0,4,0,3,3,4,4,5,0,0,4,3,0,4,5,},
243 { 0,4,0,4,0,3,0,3,0,3,4,4,4,3,3,2,4,3,4,3,4,3,5,3,4,3,2,1,4,2,4,4,3,1,3,4,2,4,5,5,3,4,5,4,1,5,4,3,0,3,2,2,3,2,1,3,1,0,3,3,3,5,3,3,3,5,4,4,2,3,3,4,3,3,3,2,1,0,3,2,1,4,3,},
244 { 0,4,0,5,0,4,0,3,0,3,5,5,3,2,4,3,4,0,5,4,4,1,4,4,4,3,3,3,4,3,5,5,2,3,3,4,1,2,5,5,3,5,5,2,3,5,5,4,0,3,2,0,3,3,1,1,5,1,4,1,0,4,3,2,3,5,0,4,0,3,0,5,4,3,4,3,0,0,4,1,0,4,4,},
245 { 1,3,0,4,0,2,0,2,0,2,5,5,3,3,3,3,3,0,4,2,3,4,4,4,3,4,0,0,3,4,5,4,3,3,3,3,2,5,5,4,5,5,5,4,3,5,5,5,1,3,1,0,1,0,0,3,2,0,4,2,0,5,2,3,2,4,1,3,0,3,0,4,5,4,5,4,3,0,4,2,0,5,4,},
246 { 0,3,0,4,0,5,0,3,0,3,4,4,3,2,3,2,3,3,3,3,3,2,4,3,3,2,2,0,3,3,3,3,3,1,3,3,3,0,4,4,3,4,4,1,1,4,4,2,0,3,1,0,1,1,0,4,1,0,2,3,1,3,3,1,3,4,0,3,0,1,0,3,1,3,0,0,1,0,2,0,0,4,4,},
247 { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,},
248 { 0,3,0,3,0,2,0,3,0,1,5,4,3,3,3,1,4,2,1,2,3,4,4,2,4,4,5,0,3,1,4,3,4,0,4,3,3,3,2,3,2,5,3,4,3,2,2,3,0,0,3,0,2,1,0,1,2,0,0,0,0,2,1,1,3,1,0,2,0,4,0,3,4,4,4,5,2,0,2,0,0,1,3,},
249 { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,1,1,0,0,1,1,0,0,0,4,2,1,1,0,1,0,3,2,0,0,3,1,1,1,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,1,0,0,0,2,0,0,0,1,4,0,4,2,1,0,0,0,0,0,1,},
250 { 0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,3,1,0,0,0,2,0,2,1,0,0,1,2,1,0,1,1,0,0,3,0,0,0,0,0,0,0,0,0,0,0,1,3,1,0,0,0,0,0,1,0,0,2,1,0,0,0,0,0,0,0,0,2,},
251 { 0,4,0,4,0,4,0,3,0,4,4,3,4,2,4,3,2,0,4,4,4,3,5,3,5,3,3,2,4,2,4,3,4,3,1,4,0,2,3,4,4,4,3,3,3,4,4,4,3,4,1,3,4,3,2,1,2,1,3,3,3,4,4,3,3,5,0,4,0,3,0,4,3,3,3,2,1,0,3,0,0,3,3,},
252 { 0,4,0,3,0,3,0,3,0,3,5,5,3,3,3,3,4,3,4,3,3,3,4,4,4,3,3,3,3,4,3,5,3,3,1,3,2,4,5,5,5,5,4,3,4,5,5,3,2,2,3,3,3,3,2,3,3,1,2,3,2,4,3,3,3,4,0,4,0,2,0,4,3,2,2,1,2,0,3,0,0,4,1,},
253 };
254 }
+93
-0
src/main/java/org/mozilla/universalchardet/prober/contextanalysis/SJISContextAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.contextanalysis;
38
39 public class SJISContextAnalysis extends JapaneseContextAnalysis {
40 ////////////////////////////////////////////////////////////////
41 // constants
42 ////////////////////////////////////////////////////////////////
43 public static final int HIRAGANA_HIGHBYTE = 0x82;
44 public static final int HIRAGANA_LOWBYTE_BEGIN = 0x9F;
45 public static final int HIRAGANA_LOWBYTE_END = 0xF1;
46 public static final int HIGHBYTE_BEGIN_1 = 0x81;
47 public static final int HIGHBYTE_END_1 = 0x9F;
48 public static final int HIGHBYTE_BEGIN_2 = 0xE0;
49 public static final int HIGHBYTE_END_2 = 0xEF;
50
51
52 ////////////////////////////////////////////////////////////////
53 // methods
54 ////////////////////////////////////////////////////////////////
55 public SJISContextAnalysis() {
56 super();
57 }
58
59 @Override
60 protected void getOrder(Order order, final byte[] buf, int offset) {
61 order.order = -1;
62 order.charLength = 1;
63
64 int highbyte = buf[offset] & 0xFF;
65 if ((highbyte >= HIGHBYTE_BEGIN_1 && highbyte <= HIGHBYTE_END_1) ||
66 (highbyte >= HIGHBYTE_BEGIN_2 && highbyte <= HIGHBYTE_END_2)) {
67 order.charLength = 2;
68 }
69
70 if (highbyte == HIRAGANA_HIGHBYTE) {
71 int lowbyte = buf[offset+1] & 0xFF;
72 if (lowbyte >= HIRAGANA_LOWBYTE_BEGIN &&
73 lowbyte <= HIRAGANA_LOWBYTE_END) {
74 order.order = (lowbyte - HIRAGANA_LOWBYTE_BEGIN);
75 }
76 }
77 }
78
79 @Override
80 protected int getOrder(final byte[] buf, int offset) {
81 int highbyte = buf[offset] & 0xFF;
82 if (highbyte == HIRAGANA_HIGHBYTE) {
83 int lowbyte = buf[offset+1] & 0xFF;
84 if (lowbyte >= HIRAGANA_LOWBYTE_BEGIN &&
85 lowbyte <= HIRAGANA_LOWBYTE_END) {
86 return (lowbyte - HIRAGANA_LOWBYTE_BEGIN);
87 }
88 }
89
90 return -1;
91 }
92 }
+962
-0
src/main/java/org/mozilla/universalchardet/prober/distributionanalysis/Big5DistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public class Big5DistributionAnalysis extends CharDistributionAnalysis {
40 ////////////////////////////////////////////////////////////////
41 // constants
42 ////////////////////////////////////////////////////////////////
43 public static final float BIG5_TYPICAL_DISTRIBUTION_RATIO = 0.75f;
44 public static final int HIGHBYTE_BEGIN = 0xA4;
45 public static final int HIGHBYTE_END = 0xFE;
46 public static final int LOWBYTE_BEGIN_1 = 0x40;
47 public static final int LOWBYTE_END_1 = 0x7E;
48 public static final int LOWBYTE_BEGIN_2 = 0xA1;
49 public static final int LOWBYTE_END_2 = 0xFE;
50
51
52 ////////////////////////////////////////////////////////////////
53 // methods
54 ////////////////////////////////////////////////////////////////
55 public Big5DistributionAnalysis() {
56 super();
57 this.charToFreqOrder = big5CharToFreqOrder;
58 this.typicalDistributionRatio = BIG5_TYPICAL_DISTRIBUTION_RATIO;
59 }
60
61 protected int getOrder(final byte[] buf, int offset) {
62 int highbyte = buf[offset] & 0xFF;
63 if (highbyte >= HIGHBYTE_BEGIN) {
64 int lowbyte = buf[offset+1] & 0xFF;
65 if (lowbyte >= LOWBYTE_BEGIN_2) {
66 return (157 * (highbyte - HIGHBYTE_BEGIN) + lowbyte - LOWBYTE_BEGIN_2 + 63);
67 } else {
68 return (157 * (highbyte - HIGHBYTE_BEGIN) + lowbyte - LOWBYTE_BEGIN_1);
69 }
70 }
71
72 return -1;
73 }
74
75
76 ////////////////////////////////////////////////////////////////
77 // constants continued
78 ////////////////////////////////////////////////////////////////
79 private static final int[] big5CharToFreqOrder = new int[] {
80 1,1801,1506, 255,1431, 198, 9, 82, 6,5008, 177, 202,3681,1256,2821, 110, // 16
81 3814, 33,3274, 261, 76, 44,2114, 16,2946,2187,1176, 659,3971, 26,3451,2653, // 32
82 1198,3972,3350,4202, 410,2215, 302, 590, 361,1964, 8, 204, 58,4510,5009,1932, // 48
83 63,5010,5011, 317,1614, 75, 222, 159,4203,2417,1480,5012,3555,3091, 224,2822, // 64
84 3682, 3, 10,3973,1471, 29,2787,1135,2866,1940, 873, 130,3275,1123, 312,5013, // 80
85 4511,2052, 507, 252, 682,5014, 142,1915, 124, 206,2947, 34,3556,3204, 64, 604, // 96
86 5015,2501,1977,1978, 155,1991, 645, 641,1606,5016,3452, 337, 72, 406,5017, 80, // 112
87 630, 238,3205,1509, 263, 939,1092,2654, 756,1440,1094,3453, 449, 69,2987, 591, // 128
88 179,2096, 471, 115,2035,1844, 60, 50,2988, 134, 806,1869, 734,2036,3454, 180, // 144
89 995,1607, 156, 537,2907, 688,5018, 319,1305, 779,2145, 514,2379, 298,4512, 359, // 160
90 2502, 90,2716,1338, 663, 11, 906,1099,2553, 20,2441, 182, 532,1716,5019, 732, // 176
91 1376,4204,1311,1420,3206, 25,2317,1056, 113, 399, 382,1950, 242,3455,2474, 529, // 192
92 3276, 475,1447,3683,5020, 117, 21, 656, 810,1297,2300,2334,3557,5021, 126,4205, // 208
93 706, 456, 150, 613,4513, 71,1118,2037,4206, 145,3092, 85, 835, 486,2115,1246, // 224
94 1426, 428, 727,1285,1015, 800, 106, 623, 303,1281,5022,2128,2359, 347,3815, 221, // 240
95 3558,3135,5023,1956,1153,4207, 83, 296,1199,3093, 192, 624, 93,5024, 822,1898, // 256
96 2823,3136, 795,2065, 991,1554,1542,1592, 27, 43,2867, 859, 139,1456, 860,4514, // 272
97 437, 712,3974, 164,2397,3137, 695, 211,3037,2097, 195,3975,1608,3559,3560,3684, // 288
98 3976, 234, 811,2989,2098,3977,2233,1441,3561,1615,2380, 668,2077,1638, 305, 228, // 304
99 1664,4515, 467, 415,5025, 262,2099,1593, 239, 108, 300, 200,1033, 512,1247,2078, // 320
100 5026,5027,2176,3207,3685,2682, 593, 845,1062,3277, 88,1723,2038,3978,1951, 212, // 336
101 266, 152, 149, 468,1899,4208,4516, 77, 187,5028,3038, 37, 5,2990,5029,3979, // 352
102 5030,5031, 39,2524,4517,2908,3208,2079, 55, 148, 74,4518, 545, 483,1474,1029, // 368
103 1665, 217,1870,1531,3138,1104,2655,4209, 24, 172,3562, 900,3980,3563,3564,4519, // 384
104 32,1408,2824,1312, 329, 487,2360,2251,2717, 784,2683, 4,3039,3351,1427,1789, // 400
105 188, 109, 499,5032,3686,1717,1790, 888,1217,3040,4520,5033,3565,5034,3352,1520, // 416
106 3687,3981, 196,1034, 775,5035,5036, 929,1816, 249, 439, 38,5037,1063,5038, 794, // 432
107 3982,1435,2301, 46, 178,3278,2066,5039,2381,5040, 214,1709,4521, 804, 35, 707, // 448
108 324,3688,1601,2554, 140, 459,4210,5041,5042,1365, 839, 272, 978,2262,2580,3456, // 464
109 2129,1363,3689,1423, 697, 100,3094, 48, 70,1231, 495,3139,2196,5043,1294,5044, // 480
110 2080, 462, 586,1042,3279, 853, 256, 988, 185,2382,3457,1698, 434,1084,5045,3458, // 496
111 314,2625,2788,4522,2335,2336, 569,2285, 637,1817,2525, 757,1162,1879,1616,3459, // 512
112 287,1577,2116, 768,4523,1671,2868,3566,2526,1321,3816, 909,2418,5046,4211, 933, // 528
113 3817,4212,2053,2361,1222,4524, 765,2419,1322, 786,4525,5047,1920,1462,1677,2909, // 544
114 1699,5048,4526,1424,2442,3140,3690,2600,3353,1775,1941,3460,3983,4213, 309,1369, // 560
115 1130,2825, 364,2234,1653,1299,3984,3567,3985,3986,2656, 525,1085,3041, 902,2001, // 576
116 1475, 964,4527, 421,1845,1415,1057,2286, 940,1364,3141, 376,4528,4529,1381, 7, // 592
117 2527, 983,2383, 336,1710,2684,1846, 321,3461, 559,1131,3042,2752,1809,1132,1313, // 608
118 265,1481,1858,5049, 352,1203,2826,3280, 167,1089, 420,2827, 776, 792,1724,3568, // 624
119 4214,2443,3281,5050,4215,5051, 446, 229, 333,2753, 901,3818,1200,1557,4530,2657, // 640
120 1921, 395,2754,2685,3819,4216,1836, 125, 916,3209,2626,4531,5052,5053,3820,5054, // 656
121 5055,5056,4532,3142,3691,1133,2555,1757,3462,1510,2318,1409,3569,5057,2146, 438, // 672
122 2601,2910,2384,3354,1068, 958,3043, 461, 311,2869,2686,4217,1916,3210,4218,1979, // 688
123 383, 750,2755,2627,4219, 274, 539, 385,1278,1442,5058,1154,1965, 384, 561, 210, // 704
124 98,1295,2556,3570,5059,1711,2420,1482,3463,3987,2911,1257, 129,5060,3821, 642, // 720
125 523,2789,2790,2658,5061, 141,2235,1333, 68, 176, 441, 876, 907,4220, 603,2602, // 736
126 710, 171,3464, 404, 549, 18,3143,2398,1410,3692,1666,5062,3571,4533,2912,4534, // 752
127 5063,2991, 368,5064, 146, 366, 99, 871,3693,1543, 748, 807,1586,1185, 22,2263, // 768
128 379,3822,3211,5065,3212, 505,1942,2628,1992,1382,2319,5066, 380,2362, 218, 702, // 784
129 1818,1248,3465,3044,3572,3355,3282,5067,2992,3694, 930,3283,3823,5068, 59,5069, // 800
130 585, 601,4221, 497,3466,1112,1314,4535,1802,5070,1223,1472,2177,5071, 749,1837, // 816
131 690,1900,3824,1773,3988,1476, 429,1043,1791,2236,2117, 917,4222, 447,1086,1629, // 832
132 5072, 556,5073,5074,2021,1654, 844,1090, 105, 550, 966,1758,2828,1008,1783, 686, // 848
133 1095,5075,2287, 793,1602,5076,3573,2603,4536,4223,2948,2302,4537,3825, 980,2503, // 864
134 544, 353, 527,4538, 908,2687,2913,5077, 381,2629,1943,1348,5078,1341,1252, 560, // 880
135 3095,5079,3467,2870,5080,2054, 973, 886,2081, 143,4539,5081,5082, 157,3989, 496, // 896
136 4224, 57, 840, 540,2039,4540,4541,3468,2118,1445, 970,2264,1748,1966,2082,4225, // 912
137 3144,1234,1776,3284,2829,3695, 773,1206,2130,1066,2040,1326,3990,1738,1725,4226, // 928
138 279,3145, 51,1544,2604, 423,1578,2131,2067, 173,4542,1880,5083,5084,1583, 264, // 944
139 610,3696,4543,2444, 280, 154,5085,5086,5087,1739, 338,1282,3096, 693,2871,1411, // 960
140 1074,3826,2445,5088,4544,5089,5090,1240, 952,2399,5091,2914,1538,2688, 685,1483, // 976
141 4227,2475,1436, 953,4228,2055,4545, 671,2400, 79,4229,2446,3285, 608, 567,2689, // 992
142 3469,4230,4231,1691, 393,1261,1792,2401,5092,4546,5093,5094,5095,5096,1383,1672, // 1008
143 3827,3213,1464, 522,1119, 661,1150, 216, 675,4547,3991,1432,3574, 609,4548,2690, // 1024
144 2402,5097,5098,5099,4232,3045, 0,5100,2476, 315, 231,2447, 301,3356,4549,2385, // 1040
145 5101, 233,4233,3697,1819,4550,4551,5102, 96,1777,1315,2083,5103, 257,5104,1810, // 1056
146 3698,2718,1139,1820,4234,2022,1124,2164,2791,1778,2659,5105,3097, 363,1655,3214, // 1072
147 5106,2993,5107,5108,5109,3992,1567,3993, 718, 103,3215, 849,1443, 341,3357,2949, // 1088
148 1484,5110,1712, 127, 67, 339,4235,2403, 679,1412, 821,5111,5112, 834, 738, 351, // 1104
149 2994,2147, 846, 235,1497,1881, 418,1993,3828,2719, 186,1100,2148,2756,3575,1545, // 1120
150 1355,2950,2872,1377, 583,3994,4236,2581,2995,5113,1298,3699,1078,2557,3700,2363, // 1136
151 78,3829,3830, 267,1289,2100,2002,1594,4237, 348, 369,1274,2197,2178,1838,4552, // 1152
152 1821,2830,3701,2757,2288,2003,4553,2951,2758, 144,3358, 882,4554,3995,2759,3470, // 1168
153 4555,2915,5114,4238,1726, 320,5115,3996,3046, 788,2996,5116,2831,1774,1327,2873, // 1184
154 3997,2832,5117,1306,4556,2004,1700,3831,3576,2364,2660, 787,2023, 506, 824,3702, // 1200
155 534, 323,4557,1044,3359,2024,1901, 946,3471,5118,1779,1500,1678,5119,1882,4558, // 1216
156 165, 243,4559,3703,2528, 123, 683,4239, 764,4560, 36,3998,1793, 589,2916, 816, // 1232
157 626,1667,3047,2237,1639,1555,1622,3832,3999,5120,4000,2874,1370,1228,1933, 891, // 1248
158 2084,2917, 304,4240,5121, 292,2997,2720,3577, 691,2101,4241,1115,4561, 118, 662, // 1264
159 5122, 611,1156, 854,2386,1316,2875, 2, 386, 515,2918,5123,5124,3286, 868,2238, // 1280
160 1486, 855,2661, 785,2216,3048,5125,1040,3216,3578,5126,3146, 448,5127,1525,5128, // 1296
161 2165,4562,5129,3833,5130,4242,2833,3579,3147, 503, 818,4001,3148,1568, 814, 676, // 1312
162 1444, 306,1749,5131,3834,1416,1030, 197,1428, 805,2834,1501,4563,5132,5133,5134, // 1328
163 1994,5135,4564,5136,5137,2198, 13,2792,3704,2998,3149,1229,1917,5138,3835,2132, // 1344
164 5139,4243,4565,2404,3580,5140,2217,1511,1727,1120,5141,5142, 646,3836,2448, 307, // 1360
165 5143,5144,1595,3217,5145,5146,5147,3705,1113,1356,4002,1465,2529,2530,5148, 519, // 1376
166 5149, 128,2133, 92,2289,1980,5150,4003,1512, 342,3150,2199,5151,2793,2218,1981, // 1392
167 3360,4244, 290,1656,1317, 789, 827,2365,5152,3837,4566, 562, 581,4004,5153, 401, // 1408
168 4567,2252, 94,4568,5154,1399,2794,5155,1463,2025,4569,3218,1944,5156, 828,1105, // 1424
169 4245,1262,1394,5157,4246, 605,4570,5158,1784,2876,5159,2835, 819,2102, 578,2200, // 1440
170 2952,5160,1502, 436,3287,4247,3288,2836,4005,2919,3472,3473,5161,2721,2320,5162, // 1456
171 5163,2337,2068, 23,4571, 193, 826,3838,2103, 699,1630,4248,3098, 390,1794,1064, // 1472
172 3581,5164,1579,3099,3100,1400,5165,4249,1839,1640,2877,5166,4572,4573, 137,4250, // 1488
173 598,3101,1967, 780, 104, 974,2953,5167, 278, 899, 253, 402, 572, 504, 493,1339, // 1504
174 5168,4006,1275,4574,2582,2558,5169,3706,3049,3102,2253, 565,1334,2722, 863, 41, // 1520
175 5170,5171,4575,5172,1657,2338, 19, 463,2760,4251, 606,5173,2999,3289,1087,2085, // 1536
176 1323,2662,3000,5174,1631,1623,1750,4252,2691,5175,2878, 791,2723,2663,2339, 232, // 1552
177 2421,5176,3001,1498,5177,2664,2630, 755,1366,3707,3290,3151,2026,1609, 119,1918, // 1568
178 3474, 862,1026,4253,5178,4007,3839,4576,4008,4577,2265,1952,2477,5179,1125, 817, // 1584
179 4254,4255,4009,1513,1766,2041,1487,4256,3050,3291,2837,3840,3152,5180,5181,1507, // 1600
180 5182,2692, 733, 40,1632,1106,2879, 345,4257, 841,2531, 230,4578,3002,1847,3292, // 1616
181 3475,5183,1263, 986,3476,5184, 735, 879, 254,1137, 857, 622,1300,1180,1388,1562, // 1632
182 4010,4011,2954, 967,2761,2665,1349, 592,2134,1692,3361,3003,1995,4258,1679,4012, // 1648
183 1902,2188,5185, 739,3708,2724,1296,1290,5186,4259,2201,2202,1922,1563,2605,2559, // 1664
184 1871,2762,3004,5187, 435,5188, 343,1108, 596, 17,1751,4579,2239,3477,3709,5189, // 1680
185 4580, 294,3582,2955,1693, 477, 979, 281,2042,3583, 643,2043,3710,2631,2795,2266, // 1696
186 1031,2340,2135,2303,3584,4581, 367,1249,2560,5190,3585,5191,4582,1283,3362,2005, // 1712
187 240,1762,3363,4583,4584, 836,1069,3153, 474,5192,2149,2532, 268,3586,5193,3219, // 1728
188 1521,1284,5194,1658,1546,4260,5195,3587,3588,5196,4261,3364,2693,1685,4262, 961, // 1744
189 1673,2632, 190,2006,2203,3841,4585,4586,5197, 570,2504,3711,1490,5198,4587,2633, // 1760
190 3293,1957,4588, 584,1514, 396,1045,1945,5199,4589,1968,2449,5200,5201,4590,4013, // 1776
191 619,5202,3154,3294, 215,2007,2796,2561,3220,4591,3221,4592, 763,4263,3842,4593, // 1792
192 5203,5204,1958,1767,2956,3365,3712,1174, 452,1477,4594,3366,3155,5205,2838,1253, // 1808
193 2387,2189,1091,2290,4264, 492,5206, 638,1169,1825,2136,1752,4014, 648, 926,1021, // 1824
194 1324,4595, 520,4596, 997, 847,1007, 892,4597,3843,2267,1872,3713,2405,1785,4598, // 1840
195 1953,2957,3103,3222,1728,4265,2044,3714,4599,2008,1701,3156,1551, 30,2268,4266, // 1856
196 5207,2027,4600,3589,5208, 501,5209,4267, 594,3478,2166,1822,3590,3479,3591,3223, // 1872
197 829,2839,4268,5210,1680,3157,1225,4269,5211,3295,4601,4270,3158,2341,5212,4602, // 1888
198 4271,5213,4015,4016,5214,1848,2388,2606,3367,5215,4603, 374,4017, 652,4272,4273, // 1904
199 375,1140, 798,5216,5217,5218,2366,4604,2269, 546,1659, 138,3051,2450,4605,5219, // 1920
200 2254, 612,1849, 910, 796,3844,1740,1371, 825,3845,3846,5220,2920,2562,5221, 692, // 1936
201 444,3052,2634, 801,4606,4274,5222,1491, 244,1053,3053,4275,4276, 340,5223,4018, // 1952
202 1041,3005, 293,1168, 87,1357,5224,1539, 959,5225,2240, 721, 694,4277,3847, 219, // 1968
203 1478, 644,1417,3368,2666,1413,1401,1335,1389,4019,5226,5227,3006,2367,3159,1826, // 1984
204 730,1515, 184,2840, 66,4607,5228,1660,2958, 246,3369, 378,1457, 226,3480, 975, // 2000
205 4020,2959,1264,3592, 674, 696,5229, 163,5230,1141,2422,2167, 713,3593,3370,4608, // 2016
206 4021,5231,5232,1186, 15,5233,1079,1070,5234,1522,3224,3594, 276,1050,2725, 758, // 2032
207 1126, 653,2960,3296,5235,2342, 889,3595,4022,3104,3007, 903,1250,4609,4023,3481, // 2048
208 3596,1342,1681,1718, 766,3297, 286, 89,2961,3715,5236,1713,5237,2607,3371,3008, // 2064
209 5238,2962,2219,3225,2880,5239,4610,2505,2533, 181, 387,1075,4024, 731,2190,3372, // 2080
210 5240,3298, 310, 313,3482,2304, 770,4278, 54,3054, 189,4611,3105,3848,4025,5241, // 2096
211 1230,1617,1850, 355,3597,4279,4612,3373, 111,4280,3716,1350,3160,3483,3055,4281, // 2112
212 2150,3299,3598,5242,2797,4026,4027,3009, 722,2009,5243,1071, 247,1207,2343,2478, // 2128
213 1378,4613,2010, 864,1437,1214,4614, 373,3849,1142,2220, 667,4615, 442,2763,2563, // 2144
214 3850,4028,1969,4282,3300,1840, 837, 170,1107, 934,1336,1883,5244,5245,2119,4283, // 2160
215 2841, 743,1569,5246,4616,4284, 582,2389,1418,3484,5247,1803,5248, 357,1395,1729, // 2176
216 3717,3301,2423,1564,2241,5249,3106,3851,1633,4617,1114,2086,4285,1532,5250, 482, // 2192
217 2451,4618,5251,5252,1492, 833,1466,5253,2726,3599,1641,2842,5254,1526,1272,3718, // 2208
218 4286,1686,1795, 416,2564,1903,1954,1804,5255,3852,2798,3853,1159,2321,5256,2881, // 2224
219 4619,1610,1584,3056,2424,2764, 443,3302,1163,3161,5257,5258,4029,5259,4287,2506, // 2240
220 3057,4620,4030,3162,2104,1647,3600,2011,1873,4288,5260,4289, 431,3485,5261, 250, // 2256
221 97, 81,4290,5262,1648,1851,1558, 160, 848,5263, 866, 740,1694,5264,2204,2843, // 2272
222 3226,4291,4621,3719,1687, 950,2479, 426, 469,3227,3720,3721,4031,5265,5266,1188, // 2288
223 424,1996, 861,3601,4292,3854,2205,2694, 168,1235,3602,4293,5267,2087,1674,4622, // 2304
224 3374,3303, 220,2565,1009,5268,3855, 670,3010, 332,1208, 717,5269,5270,3603,2452, // 2320
225 4032,3375,5271, 513,5272,1209,2882,3376,3163,4623,1080,5273,5274,5275,5276,2534, // 2336
226 3722,3604, 815,1587,4033,4034,5277,3605,3486,3856,1254,4624,1328,3058,1390,4035, // 2352
227 1741,4036,3857,4037,5278, 236,3858,2453,3304,5279,5280,3723,3859,1273,3860,4625, // 2368
228 5281, 308,5282,4626, 245,4627,1852,2480,1307,2583, 430, 715,2137,2454,5283, 270, // 2384
229 199,2883,4038,5284,3606,2727,1753, 761,1754, 725,1661,1841,4628,3487,3724,5285, // 2400
230 5286, 587, 14,3305, 227,2608, 326, 480,2270, 943,2765,3607, 291, 650,1884,5287, // 2416
231 1702,1226, 102,1547, 62,3488, 904,4629,3489,1164,4294,5288,5289,1224,1548,2766, // 2432
232 391, 498,1493,5290,1386,1419,5291,2056,1177,4630, 813, 880,1081,2368, 566,1145, // 2448
233 4631,2291,1001,1035,2566,2609,2242, 394,1286,5292,5293,2069,5294, 86,1494,1730, // 2464
234 4039, 491,1588, 745, 897,2963, 843,3377,4040,2767,2884,3306,1768, 998,2221,2070, // 2480
235 397,1827,1195,1970,3725,3011,3378, 284,5295,3861,2507,2138,2120,1904,5296,4041, // 2496
236 2151,4042,4295,1036,3490,1905, 114,2567,4296, 209,1527,5297,5298,2964,2844,2635, // 2512
237 2390,2728,3164, 812,2568,5299,3307,5300,1559, 737,1885,3726,1210, 885, 28,2695, // 2528
238 3608,3862,5301,4297,1004,1780,4632,5302, 346,1982,2222,2696,4633,3863,1742, 797, // 2544
239 1642,4043,1934,1072,1384,2152, 896,4044,3308,3727,3228,2885,3609,5303,2569,1959, // 2560
240 4634,2455,1786,5304,5305,5306,4045,4298,1005,1308,3728,4299,2729,4635,4636,1528, // 2576
241 2610, 161,1178,4300,1983, 987,4637,1101,4301, 631,4046,1157,3229,2425,1343,1241, // 2592
242 1016,2243,2570, 372, 877,2344,2508,1160, 555,1935, 911,4047,5307, 466,1170, 169, // 2608
243 1051,2921,2697,3729,2481,3012,1182,2012,2571,1251,2636,5308, 992,2345,3491,1540, // 2624
244 2730,1201,2071,2406,1997,2482,5309,4638, 528,1923,2191,1503,1874,1570,2369,3379, // 2640
245 3309,5310, 557,1073,5311,1828,3492,2088,2271,3165,3059,3107, 767,3108,2799,4639, // 2656
246 1006,4302,4640,2346,1267,2179,3730,3230, 778,4048,3231,2731,1597,2667,5312,4641, // 2672
247 5313,3493,5314,5315,5316,3310,2698,1433,3311, 131, 95,1504,4049, 723,4303,3166, // 2688
248 1842,3610,2768,2192,4050,2028,2105,3731,5317,3013,4051,1218,5318,3380,3232,4052, // 2704
249 4304,2584, 248,1634,3864, 912,5319,2845,3732,3060,3865, 654, 53,5320,3014,5321, // 2720
250 1688,4642, 777,3494,1032,4053,1425,5322, 191, 820,2121,2846, 971,4643, 931,3233, // 2736
251 135, 664, 783,3866,1998, 772,2922,1936,4054,3867,4644,2923,3234, 282,2732, 640, // 2752
252 1372,3495,1127, 922, 325,3381,5323,5324, 711,2045,5325,5326,4055,2223,2800,1937, // 2768
253 4056,3382,2224,2255,3868,2305,5327,4645,3869,1258,3312,4057,3235,2139,2965,4058, // 2784
254 4059,5328,2225, 258,3236,4646, 101,1227,5329,3313,1755,5330,1391,3314,5331,2924, // 2800
255 2057, 893,5332,5333,5334,1402,4305,2347,5335,5336,3237,3611,5337,5338, 878,1325, // 2816
256 1781,2801,4647, 259,1385,2585, 744,1183,2272,4648,5339,4060,2509,5340, 684,1024, // 2832
257 4306,5341, 472,3612,3496,1165,3315,4061,4062, 322,2153, 881, 455,1695,1152,1340, // 2848
258 660, 554,2154,4649,1058,4650,4307, 830,1065,3383,4063,4651,1924,5342,1703,1919, // 2864
259 5343, 932,2273, 122,5344,4652, 947, 677,5345,3870,2637, 297,1906,1925,2274,4653, // 2880
260 2322,3316,5346,5347,4308,5348,4309, 84,4310, 112, 989,5349, 547,1059,4064, 701, // 2896
261 3613,1019,5350,4311,5351,3497, 942, 639, 457,2306,2456, 993,2966, 407, 851, 494, // 2912
262 4654,3384, 927,5352,1237,5353,2426,3385, 573,4312, 680, 921,2925,1279,1875, 285, // 2928
263 790,1448,1984, 719,2168,5354,5355,4655,4065,4066,1649,5356,1541, 563,5357,1077, // 2944
264 5358,3386,3061,3498, 511,3015,4067,4068,3733,4069,1268,2572,3387,3238,4656,4657, // 2960
265 5359, 535,1048,1276,1189,2926,2029,3167,1438,1373,2847,2967,1134,2013,5360,4313, // 2976
266 1238,2586,3109,1259,5361, 700,5362,2968,3168,3734,4314,5363,4315,1146,1876,1907, // 2992
267 4658,2611,4070, 781,2427, 132,1589, 203, 147, 273,2802,2407, 898,1787,2155,4071, // 3008
268 4072,5364,3871,2803,5365,5366,4659,4660,5367,3239,5368,1635,3872, 965,5369,1805, // 3024
269 2699,1516,3614,1121,1082,1329,3317,4073,1449,3873, 65,1128,2848,2927,2769,1590, // 3040
270 3874,5370,5371, 12,2668, 45, 976,2587,3169,4661, 517,2535,1013,1037,3240,5372, // 3056
271 3875,2849,5373,3876,5374,3499,5375,2612, 614,1999,2323,3877,3110,2733,2638,5376, // 3072
272 2588,4316, 599,1269,5377,1811,3735,5378,2700,3111, 759,1060, 489,1806,3388,3318, // 3088
273 1358,5379,5380,2391,1387,1215,2639,2256, 490,5381,5382,4317,1759,2392,2348,5383, // 3104
274 4662,3878,1908,4074,2640,1807,3241,4663,3500,3319,2770,2349, 874,5384,5385,3501, // 3120
275 3736,1859, 91,2928,3737,3062,3879,4664,5386,3170,4075,2669,5387,3502,1202,1403, // 3136
276 3880,2969,2536,1517,2510,4665,3503,2511,5388,4666,5389,2701,1886,1495,1731,4076, // 3152
277 2370,4667,5390,2030,5391,5392,4077,2702,1216, 237,2589,4318,2324,4078,3881,4668, // 3168
278 4669,2703,3615,3504, 445,4670,5393,5394,5395,5396,2771, 61,4079,3738,1823,4080, // 3184
279 5397, 687,2046, 935, 925, 405,2670, 703,1096,1860,2734,4671,4081,1877,1367,2704, // 3200
280 3389, 918,2106,1782,2483, 334,3320,1611,1093,4672, 564,3171,3505,3739,3390, 945, // 3216
281 2641,2058,4673,5398,1926, 872,4319,5399,3506,2705,3112, 349,4320,3740,4082,4674, // 3232
282 3882,4321,3741,2156,4083,4675,4676,4322,4677,2408,2047, 782,4084, 400, 251,4323, // 3248
283 1624,5400,5401, 277,3742, 299,1265, 476,1191,3883,2122,4324,4325,1109, 205,5402, // 3264
284 2590,1000,2157,3616,1861,5403,5404,5405,4678,5406,4679,2573, 107,2484,2158,4085, // 3280
285 3507,3172,5407,1533, 541,1301, 158, 753,4326,2886,3617,5408,1696, 370,1088,4327, // 3296
286 4680,3618, 579, 327, 440, 162,2244, 269,1938,1374,3508, 968,3063, 56,1396,3113, // 3312
287 2107,3321,3391,5409,1927,2159,4681,3016,5410,3619,5411,5412,3743,4682,2485,5413, // 3328
288 2804,5414,1650,4683,5415,2613,5416,5417,4086,2671,3392,1149,3393,4087,3884,4088, // 3344
289 5418,1076, 49,5419, 951,3242,3322,3323, 450,2850, 920,5420,1812,2805,2371,4328, // 3360
290 1909,1138,2372,3885,3509,5421,3243,4684,1910,1147,1518,2428,4685,3886,5422,4686, // 3376
291 2393,2614, 260,1796,3244,5423,5424,3887,3324, 708,5425,3620,1704,5426,3621,1351, // 3392
292 1618,3394,3017,1887, 944,4329,3395,4330,3064,3396,4331,5427,3744, 422, 413,1714, // 3408
293 3325, 500,2059,2350,4332,2486,5428,1344,1911, 954,5429,1668,5430,5431,4089,2409, // 3424
294 4333,3622,3888,4334,5432,2307,1318,2512,3114, 133,3115,2887,4687, 629, 31,2851, // 3440
295 2706,3889,4688, 850, 949,4689,4090,2970,1732,2089,4335,1496,1853,5433,4091, 620, // 3456
296 3245, 981,1242,3745,3397,1619,3746,1643,3326,2140,2457,1971,1719,3510,2169,5434, // 3472
297 3246,5435,5436,3398,1829,5437,1277,4690,1565,2048,5438,1636,3623,3116,5439, 869, // 3488
298 2852, 655,3890,3891,3117,4092,3018,3892,1310,3624,4691,5440,5441,5442,1733, 558, // 3504
299 4692,3747, 335,1549,3065,1756,4336,3748,1946,3511,1830,1291,1192, 470,2735,2108, // 3520
300 2806, 913,1054,4093,5443,1027,5444,3066,4094,4693, 982,2672,3399,3173,3512,3247, // 3536
301 3248,1947,2807,5445, 571,4694,5446,1831,5447,3625,2591,1523,2429,5448,2090, 984, // 3552
302 4695,3749,1960,5449,3750, 852, 923,2808,3513,3751, 969,1519, 999,2049,2325,1705, // 3568
303 5450,3118, 615,1662, 151, 597,4095,2410,2326,1049, 275,4696,3752,4337, 568,3753, // 3584
304 3626,2487,4338,3754,5451,2430,2275, 409,3249,5452,1566,2888,3514,1002, 769,2853, // 3600
305 194,2091,3174,3755,2226,3327,4339, 628,1505,5453,5454,1763,2180,3019,4096, 521, // 3616
306 1161,2592,1788,2206,2411,4697,4097,1625,4340,4341, 412, 42,3119, 464,5455,2642, // 3632
307 4698,3400,1760,1571,2889,3515,2537,1219,2207,3893,2643,2141,2373,4699,4700,3328, // 3648
308 1651,3401,3627,5456,5457,3628,2488,3516,5458,3756,5459,5460,2276,2092, 460,5461, // 3664
309 4701,5462,3020, 962, 588,3629, 289,3250,2644,1116, 52,5463,3067,1797,5464,5465, // 3680
310 5466,1467,5467,1598,1143,3757,4342,1985,1734,1067,4702,1280,3402, 465,4703,1572, // 3696
311 510,5468,1928,2245,1813,1644,3630,5469,4704,3758,5470,5471,2673,1573,1534,5472, // 3712
312 5473, 536,1808,1761,3517,3894,3175,2645,5474,5475,5476,4705,3518,2929,1912,2809, // 3728
313 5477,3329,1122, 377,3251,5478, 360,5479,5480,4343,1529, 551,5481,2060,3759,1769, // 3744
314 2431,5482,2930,4344,3330,3120,2327,2109,2031,4706,1404, 136,1468,1479, 672,1171, // 3760
315 3252,2308, 271,3176,5483,2772,5484,2050, 678,2736, 865,1948,4707,5485,2014,4098, // 3776
316 2971,5486,2737,2227,1397,3068,3760,4708,4709,1735,2931,3403,3631,5487,3895, 509, // 3792
317 2854,2458,2890,3896,5488,5489,3177,3178,4710,4345,2538,4711,2309,1166,1010, 552, // 3808
318 681,1888,5490,5491,2972,2973,4099,1287,1596,1862,3179, 358, 453, 736, 175, 478, // 3824
319 1117, 905,1167,1097,5492,1854,1530,5493,1706,5494,2181,3519,2292,3761,3520,3632, // 3840
320 4346,2093,4347,5495,3404,1193,2489,4348,1458,2193,2208,1863,1889,1421,3331,2932, // 3856
321 3069,2182,3521, 595,2123,5496,4100,5497,5498,4349,1707,2646, 223,3762,1359, 751, // 3872
322 3121, 183,3522,5499,2810,3021, 419,2374, 633, 704,3897,2394, 241,5500,5501,5502, // 3888
323 838,3022,3763,2277,2773,2459,3898,1939,2051,4101,1309,3122,2246,1181,5503,1136, // 3904
324 2209,3899,2375,1446,4350,2310,4712,5504,5505,4351,1055,2615, 484,3764,5506,4102, // 3920
325 625,4352,2278,3405,1499,4353,4103,5507,4104,4354,3253,2279,2280,3523,5508,5509, // 3936
326 2774, 808,2616,3765,3406,4105,4355,3123,2539, 526,3407,3900,4356, 955,5510,1620, // 3952
327 4357,2647,2432,5511,1429,3766,1669,1832, 994, 928,5512,3633,1260,5513,5514,5515, // 3968
328 1949,2293, 741,2933,1626,4358,2738,2460, 867,1184, 362,3408,1392,5516,5517,4106, // 3984
329 4359,1770,1736,3254,2934,4713,4714,1929,2707,1459,1158,5518,3070,3409,2891,1292, // 4000
330 1930,2513,2855,3767,1986,1187,2072,2015,2617,4360,5519,2574,2514,2170,3768,2490, // 4016
331 3332,5520,3769,4715,5521,5522, 666,1003,3023,1022,3634,4361,5523,4716,1814,2257, // 4032
332 574,3901,1603, 295,1535, 705,3902,4362, 283, 858, 417,5524,5525,3255,4717,4718, // 4048
333 3071,1220,1890,1046,2281,2461,4107,1393,1599, 689,2575, 388,4363,5526,2491, 802, // 4064
334 5527,2811,3903,2061,1405,2258,5528,4719,3904,2110,1052,1345,3256,1585,5529, 809, // 4080
335 5530,5531,5532, 575,2739,3524, 956,1552,1469,1144,2328,5533,2329,1560,2462,3635, // 4096
336 3257,4108, 616,2210,4364,3180,2183,2294,5534,1833,5535,3525,4720,5536,1319,3770, // 4112
337 3771,1211,3636,1023,3258,1293,2812,5537,5538,5539,3905, 607,2311,3906, 762,2892, // 4128
338 1439,4365,1360,4721,1485,3072,5540,4722,1038,4366,1450,2062,2648,4367,1379,4723, // 4144
339 2593,5541,5542,4368,1352,1414,2330,2935,1172,5543,5544,3907,3908,4724,1798,1451, // 4160
340 5545,5546,5547,5548,2936,4109,4110,2492,2351, 411,4111,4112,3637,3333,3124,4725, // 4176
341 1561,2674,1452,4113,1375,5549,5550, 47,2974, 316,5551,1406,1591,2937,3181,5552, // 4192
342 1025,2142,3125,3182, 354,2740, 884,2228,4369,2412, 508,3772, 726,3638, 996,2433, // 4208
343 3639, 729,5553, 392,2194,1453,4114,4726,3773,5554,5555,2463,3640,2618,1675,2813, // 4224
344 919,2352,2975,2353,1270,4727,4115, 73,5556,5557, 647,5558,3259,2856,2259,1550, // 4240
345 1346,3024,5559,1332, 883,3526,5560,5561,5562,5563,3334,2775,5564,1212, 831,1347, // 4256
346 4370,4728,2331,3909,1864,3073, 720,3910,4729,4730,3911,5565,4371,5566,5567,4731, // 4272
347 5568,5569,1799,4732,3774,2619,4733,3641,1645,2376,4734,5570,2938, 669,2211,2675, // 4288
348 2434,5571,2893,5572,5573,1028,3260,5574,4372,2413,5575,2260,1353,5576,5577,4735, // 4304
349 3183, 518,5578,4116,5579,4373,1961,5580,2143,4374,5581,5582,3025,2354,2355,3912, // 4320
350 516,1834,1454,4117,2708,4375,4736,2229,2620,1972,1129,3642,5583,2776,5584,2976, // 4336
351 1422, 577,1470,3026,1524,3410,5585,5586, 432,4376,3074,3527,5587,2594,1455,2515, // 4352
352 2230,1973,1175,5588,1020,2741,4118,3528,4737,5589,2742,5590,1743,1361,3075,3529, // 4368
353 2649,4119,4377,4738,2295, 895, 924,4378,2171, 331,2247,3076, 166,1627,3077,1098, // 4384
354 5591,1232,2894,2231,3411,4739, 657, 403,1196,2377, 542,3775,3412,1600,4379,3530, // 4400
355 5592,4740,2777,3261, 576, 530,1362,4741,4742,2540,2676,3776,4120,5593, 842,3913, // 4416
356 5594,2814,2032,1014,4121, 213,2709,3413, 665, 621,4380,5595,3777,2939,2435,5596, // 4432
357 2436,3335,3643,3414,4743,4381,2541,4382,4744,3644,1682,4383,3531,1380,5597, 724, // 4448
358 2282, 600,1670,5598,1337,1233,4745,3126,2248,5599,1621,4746,5600, 651,4384,5601, // 4464
359 1612,4385,2621,5602,2857,5603,2743,2312,3078,5604, 716,2464,3079, 174,1255,2710, // 4480
360 4122,3645, 548,1320,1398, 728,4123,1574,5605,1891,1197,3080,4124,5606,3081,3082, // 4496
361 3778,3646,3779, 747,5607, 635,4386,4747,5608,5609,5610,4387,5611,5612,4748,5613, // 4512
362 3415,4749,2437, 451,5614,3780,2542,2073,4388,2744,4389,4125,5615,1764,4750,5616, // 4528
363 4390, 350,4751,2283,2395,2493,5617,4391,4126,2249,1434,4127, 488,4752, 458,4392, // 4544
364 4128,3781, 771,1330,2396,3914,2576,3184,2160,2414,1553,2677,3185,4393,5618,2494, // 4560
365 2895,2622,1720,2711,4394,3416,4753,5619,2543,4395,5620,3262,4396,2778,5621,2016, // 4576
366 2745,5622,1155,1017,3782,3915,5623,3336,2313, 201,1865,4397,1430,5624,4129,5625, // 4592
367 5626,5627,5628,5629,4398,1604,5630, 414,1866, 371,2595,4754,4755,3532,2017,3127, // 4608
368 4756,1708, 960,4399, 887, 389,2172,1536,1663,1721,5631,2232,4130,2356,2940,1580, // 4624
369 5632,5633,1744,4757,2544,4758,4759,5634,4760,5635,2074,5636,4761,3647,3417,2896, // 4640
370 4400,5637,4401,2650,3418,2815, 673,2712,2465, 709,3533,4131,3648,4402,5638,1148, // 4656
371 502, 634,5639,5640,1204,4762,3649,1575,4763,2623,3783,5641,3784,3128, 948,3263, // 4672
372 121,1745,3916,1110,5642,4403,3083,2516,3027,4132,3785,1151,1771,3917,1488,4133, // 4688
373 1987,5643,2438,3534,5644,5645,2094,5646,4404,3918,1213,1407,2816, 531,2746,2545, // 4704
374 3264,1011,1537,4764,2779,4405,3129,1061,5647,3786,3787,1867,2897,5648,2018, 120, // 4720
375 4406,4407,2063,3650,3265,2314,3919,2678,3419,1955,4765,4134,5649,3535,1047,2713, // 4736
376 1266,5650,1368,4766,2858, 649,3420,3920,2546,2747,1102,2859,2679,5651,5652,2000, // 4752
377 5653,1111,3651,2977,5654,2495,3921,3652,2817,1855,3421,3788,5655,5656,3422,2415, // 4768
378 2898,3337,3266,3653,5657,2577,5658,3654,2818,4135,1460, 856,5659,3655,5660,2899, // 4784
379 2978,5661,2900,3922,5662,4408, 632,2517, 875,3923,1697,3924,2296,5663,5664,4767, // 4800
380 3028,1239, 580,4768,4409,5665, 914, 936,2075,1190,4136,1039,2124,5666,5667,5668, // 4816
381 5669,3423,1473,5670,1354,4410,3925,4769,2173,3084,4137, 915,3338,4411,4412,3339, // 4832
382 1605,1835,5671,2748, 398,3656,4413,3926,4138, 328,1913,2860,4139,3927,1331,4414, // 4848
383 3029, 937,4415,5672,3657,4140,4141,3424,2161,4770,3425, 524, 742, 538,3085,1012, // 4864
384 5673,5674,3928,2466,5675, 658,1103, 225,3929,5676,5677,4771,5678,4772,5679,3267, // 4880
385 1243,5680,4142, 963,2250,4773,5681,2714,3658,3186,5682,5683,2596,2332,5684,4774, // 4896
386 5685,5686,5687,3536, 957,3426,2547,2033,1931,2941,2467, 870,2019,3659,1746,2780, // 4912
387 2781,2439,2468,5688,3930,5689,3789,3130,3790,3537,3427,3791,5690,1179,3086,5691, // 4928
388 3187,2378,4416,3792,2548,3188,3131,2749,4143,5692,3428,1556,2549,2297, 977,2901, // 4944
389 2034,4144,1205,3429,5693,1765,3430,3189,2125,1271, 714,1689,4775,3538,5694,2333, // 4960
390 3931, 533,4417,3660,2184, 617,5695,2469,3340,3539,2315,5696,5697,3190,5698,5699, // 4976
391 3932,1988, 618, 427,2651,3540,3431,5700,5701,1244,1690,5702,2819,4418,4776,5703, // 4992
392 3541,4777,5704,2284,1576, 473,3661,4419,3432, 972,5705,3662,5706,3087,5707,5708, // 5008
393 4778,4779,5709,3793,4145,4146,5710, 153,4780, 356,5711,1892,2902,4420,2144, 408, // 5024
394 803,2357,5712,3933,5713,4421,1646,2578,2518,4781,4782,3934,5714,3935,4422,5715, // 5040
395 2416,3433, 752,5716,5717,1962,3341,2979,5718, 746,3030,2470,4783,4423,3794, 698, // 5056
396 4784,1893,4424,3663,2550,4785,3664,3936,5719,3191,3434,5720,1824,1302,4147,2715, // 5072
397 3937,1974,4425,5721,4426,3192, 823,1303,1288,1236,2861,3542,4148,3435, 774,3938, // 5088
398 5722,1581,4786,1304,2862,3939,4787,5723,2440,2162,1083,3268,4427,4149,4428, 344, // 5104
399 1173, 288,2316, 454,1683,5724,5725,1461,4788,4150,2597,5726,5727,4789, 985, 894, // 5120
400 5728,3436,3193,5729,1914,2942,3795,1989,5730,2111,1975,5731,4151,5732,2579,1194, // 5136
401 425,5733,4790,3194,1245,3796,4429,5734,5735,2863,5736, 636,4791,1856,3940, 760, // 5152
402 1800,5737,4430,2212,1508,4792,4152,1894,1684,2298,5738,5739,4793,4431,4432,2213, // 5168
403 479,5740,5741, 832,5742,4153,2496,5743,2980,2497,3797, 990,3132, 627,1815,2652, // 5184
404 4433,1582,4434,2126,2112,3543,4794,5744, 799,4435,3195,5745,4795,2113,1737,3031, // 5200
405 1018, 543, 754,4436,3342,1676,4796,4797,4154,4798,1489,5746,3544,5747,2624,2903, // 5216
406 4155,5748,5749,2981,5750,5751,5752,5753,3196,4799,4800,2185,1722,5754,3269,3270, // 5232
407 1843,3665,1715, 481, 365,1976,1857,5755,5756,1963,2498,4801,5757,2127,3666,3271, // 5248
408 433,1895,2064,2076,5758, 602,2750,5759,5760,5761,5762,5763,3032,1628,3437,5764, // 5264
409 3197,4802,4156,2904,4803,2519,5765,2551,2782,5766,5767,5768,3343,4804,2905,5769, // 5280
410 4805,5770,2864,4806,4807,1221,2982,4157,2520,5771,5772,5773,1868,1990,5774,5775, // 5296
411 5776,1896,5777,5778,4808,1897,4158, 318,5779,2095,4159,4437,5780,5781, 485,5782, // 5312
412 938,3941, 553,2680, 116,5783,3942,3667,5784,3545,2681,2783,3438,3344,2820,5785, // 5328
413 3668,2943,4160,1747,2944,2983,5786,5787, 207,5788,4809,5789,4810,2521,5790,3033, // 5344
414 890,3669,3943,5791,1878,3798,3439,5792,2186,2358,3440,1652,5793,5794,5795, 941, // 5360
415 2299, 208,3546,4161,2020, 330,4438,3944,2906,2499,3799,4439,4811,5796,5797,5798, // 5376 //last 512
416
417 /***************************************************************************************
418 *Everything below is of no interest for detection purpose *
419 ***************************************************************************************
420
421 2522,1613,4812,5799,3345,3945,2523,5800,4162,5801,1637,4163,2471,4813,3946,5802, // 5392
422 2500,3034,3800,5803,5804,2195,4814,5805,2163,5806,5807,5808,5809,5810,5811,5812, // 5408
423 5813,5814,5815,5816,5817,5818,5819,5820,5821,5822,5823,5824,5825,5826,5827,5828, // 5424
424 5829,5830,5831,5832,5833,5834,5835,5836,5837,5838,5839,5840,5841,5842,5843,5844, // 5440
425 5845,5846,5847,5848,5849,5850,5851,5852,5853,5854,5855,5856,5857,5858,5859,5860, // 5456
426 5861,5862,5863,5864,5865,5866,5867,5868,5869,5870,5871,5872,5873,5874,5875,5876, // 5472
427 5877,5878,5879,5880,5881,5882,5883,5884,5885,5886,5887,5888,5889,5890,5891,5892, // 5488
428 5893,5894,5895,5896,5897,5898,5899,5900,5901,5902,5903,5904,5905,5906,5907,5908, // 5504
429 5909,5910,5911,5912,5913,5914,5915,5916,5917,5918,5919,5920,5921,5922,5923,5924, // 5520
430 5925,5926,5927,5928,5929,5930,5931,5932,5933,5934,5935,5936,5937,5938,5939,5940, // 5536
431 5941,5942,5943,5944,5945,5946,5947,5948,5949,5950,5951,5952,5953,5954,5955,5956, // 5552
432 5957,5958,5959,5960,5961,5962,5963,5964,5965,5966,5967,5968,5969,5970,5971,5972, // 5568
433 5973,5974,5975,5976,5977,5978,5979,5980,5981,5982,5983,5984,5985,5986,5987,5988, // 5584
434 5989,5990,5991,5992,5993,5994,5995,5996,5997,5998,5999,6000,6001,6002,6003,6004, // 5600
435 6005,6006,6007,6008,6009,6010,6011,6012,6013,6014,6015,6016,6017,6018,6019,6020, // 5616
436 6021,6022,6023,6024,6025,6026,6027,6028,6029,6030,6031,6032,6033,6034,6035,6036, // 5632
437 6037,6038,6039,6040,6041,6042,6043,6044,6045,6046,6047,6048,6049,6050,6051,6052, // 5648
438 6053,6054,6055,6056,6057,6058,6059,6060,6061,6062,6063,6064,6065,6066,6067,6068, // 5664
439 6069,6070,6071,6072,6073,6074,6075,6076,6077,6078,6079,6080,6081,6082,6083,6084, // 5680
440 6085,6086,6087,6088,6089,6090,6091,6092,6093,6094,6095,6096,6097,6098,6099,6100, // 5696
441 6101,6102,6103,6104,6105,6106,6107,6108,6109,6110,6111,6112,6113,6114,6115,6116, // 5712
442 6117,6118,6119,6120,6121,6122,6123,6124,6125,6126,6127,6128,6129,6130,6131,6132, // 5728
443 6133,6134,6135,6136,6137,6138,6139,6140,6141,6142,6143,6144,6145,6146,6147,6148, // 5744
444 6149,6150,6151,6152,6153,6154,6155,6156,6157,6158,6159,6160,6161,6162,6163,6164, // 5760
445 6165,6166,6167,6168,6169,6170,6171,6172,6173,6174,6175,6176,6177,6178,6179,6180, // 5776
446 6181,6182,6183,6184,6185,6186,6187,6188,6189,6190,6191,6192,6193,6194,6195,6196, // 5792
447 6197,6198,6199,6200,6201,6202,6203,6204,6205,6206,6207,6208,6209,6210,6211,6212, // 5808
448 6213,6214,6215,6216,6217,6218,6219,6220,6221,6222,6223,3670,6224,6225,6226,6227, // 5824
449 6228,6229,6230,6231,6232,6233,6234,6235,6236,6237,6238,6239,6240,6241,6242,6243, // 5840
450 6244,6245,6246,6247,6248,6249,6250,6251,6252,6253,6254,6255,6256,6257,6258,6259, // 5856
451 6260,6261,6262,6263,6264,6265,6266,6267,6268,6269,6270,6271,6272,6273,6274,6275, // 5872
452 6276,6277,6278,6279,6280,6281,6282,6283,6284,6285,4815,6286,6287,6288,6289,6290, // 5888
453 6291,6292,4816,6293,6294,6295,6296,6297,6298,6299,6300,6301,6302,6303,6304,6305, // 5904
454 6306,6307,6308,6309,6310,6311,4817,4818,6312,6313,6314,6315,6316,6317,6318,4819, // 5920
455 6319,6320,6321,6322,6323,6324,6325,6326,6327,6328,6329,6330,6331,6332,6333,6334, // 5936
456 6335,6336,6337,4820,6338,6339,6340,6341,6342,6343,6344,6345,6346,6347,6348,6349, // 5952
457 6350,6351,6352,6353,6354,6355,6356,6357,6358,6359,6360,6361,6362,6363,6364,6365, // 5968
458 6366,6367,6368,6369,6370,6371,6372,6373,6374,6375,6376,6377,6378,6379,6380,6381, // 5984
459 6382,6383,6384,6385,6386,6387,6388,6389,6390,6391,6392,6393,6394,6395,6396,6397, // 6000
460 6398,6399,6400,6401,6402,6403,6404,6405,6406,6407,6408,6409,6410,3441,6411,6412, // 6016
461 6413,6414,6415,6416,6417,6418,6419,6420,6421,6422,6423,6424,6425,4440,6426,6427, // 6032
462 6428,6429,6430,6431,6432,6433,6434,6435,6436,6437,6438,6439,6440,6441,6442,6443, // 6048
463 6444,6445,6446,6447,6448,6449,6450,6451,6452,6453,6454,4821,6455,6456,6457,6458, // 6064
464 6459,6460,6461,6462,6463,6464,6465,6466,6467,6468,6469,6470,6471,6472,6473,6474, // 6080
465 6475,6476,6477,3947,3948,6478,6479,6480,6481,3272,4441,6482,6483,6484,6485,4442, // 6096
466 6486,6487,6488,6489,6490,6491,6492,6493,6494,6495,6496,4822,6497,6498,6499,6500, // 6112
467 6501,6502,6503,6504,6505,6506,6507,6508,6509,6510,6511,6512,6513,6514,6515,6516, // 6128
468 6517,6518,6519,6520,6521,6522,6523,6524,6525,6526,6527,6528,6529,6530,6531,6532, // 6144
469 6533,6534,6535,6536,6537,6538,6539,6540,6541,6542,6543,6544,6545,6546,6547,6548, // 6160
470 6549,6550,6551,6552,6553,6554,6555,6556,2784,6557,4823,6558,6559,6560,6561,6562, // 6176
471 6563,6564,6565,6566,6567,6568,6569,3949,6570,6571,6572,4824,6573,6574,6575,6576, // 6192
472 6577,6578,6579,6580,6581,6582,6583,4825,6584,6585,6586,3950,2785,6587,6588,6589, // 6208
473 6590,6591,6592,6593,6594,6595,6596,6597,6598,6599,6600,6601,6602,6603,6604,6605, // 6224
474 6606,6607,6608,6609,6610,6611,6612,4826,6613,6614,6615,4827,6616,6617,6618,6619, // 6240
475 6620,6621,6622,6623,6624,6625,4164,6626,6627,6628,6629,6630,6631,6632,6633,6634, // 6256
476 3547,6635,4828,6636,6637,6638,6639,6640,6641,6642,3951,2984,6643,6644,6645,6646, // 6272
477 6647,6648,6649,4165,6650,4829,6651,6652,4830,6653,6654,6655,6656,6657,6658,6659, // 6288
478 6660,6661,6662,4831,6663,6664,6665,6666,6667,6668,6669,6670,6671,4166,6672,4832, // 6304
479 3952,6673,6674,6675,6676,4833,6677,6678,6679,4167,6680,6681,6682,3198,6683,6684, // 6320
480 6685,6686,6687,6688,6689,6690,6691,6692,6693,6694,6695,6696,6697,4834,6698,6699, // 6336
481 6700,6701,6702,6703,6704,6705,6706,6707,6708,6709,6710,6711,6712,6713,6714,6715, // 6352
482 6716,6717,6718,6719,6720,6721,6722,6723,6724,6725,6726,6727,6728,6729,6730,6731, // 6368
483 6732,6733,6734,4443,6735,6736,6737,6738,6739,6740,6741,6742,6743,6744,6745,4444, // 6384
484 6746,6747,6748,6749,6750,6751,6752,6753,6754,6755,6756,6757,6758,6759,6760,6761, // 6400
485 6762,6763,6764,6765,6766,6767,6768,6769,6770,6771,6772,6773,6774,6775,6776,6777, // 6416
486 6778,6779,6780,6781,4168,6782,6783,3442,6784,6785,6786,6787,6788,6789,6790,6791, // 6432
487 4169,6792,6793,6794,6795,6796,6797,6798,6799,6800,6801,6802,6803,6804,6805,6806, // 6448
488 6807,6808,6809,6810,6811,4835,6812,6813,6814,4445,6815,6816,4446,6817,6818,6819, // 6464
489 6820,6821,6822,6823,6824,6825,6826,6827,6828,6829,6830,6831,6832,6833,6834,6835, // 6480
490 3548,6836,6837,6838,6839,6840,6841,6842,6843,6844,6845,6846,4836,6847,6848,6849, // 6496
491 6850,6851,6852,6853,6854,3953,6855,6856,6857,6858,6859,6860,6861,6862,6863,6864, // 6512
492 6865,6866,6867,6868,6869,6870,6871,6872,6873,6874,6875,6876,6877,3199,6878,6879, // 6528
493 6880,6881,6882,4447,6883,6884,6885,6886,6887,6888,6889,6890,6891,6892,6893,6894, // 6544
494 6895,6896,6897,6898,6899,6900,6901,6902,6903,6904,4170,6905,6906,6907,6908,6909, // 6560
495 6910,6911,6912,6913,6914,6915,6916,6917,6918,6919,6920,6921,6922,6923,6924,6925, // 6576
496 6926,6927,4837,6928,6929,6930,6931,6932,6933,6934,6935,6936,3346,6937,6938,4838, // 6592
497 6939,6940,6941,4448,6942,6943,6944,6945,6946,4449,6947,6948,6949,6950,6951,6952, // 6608
498 6953,6954,6955,6956,6957,6958,6959,6960,6961,6962,6963,6964,6965,6966,6967,6968, // 6624
499 6969,6970,6971,6972,6973,6974,6975,6976,6977,6978,6979,6980,6981,6982,6983,6984, // 6640
500 6985,6986,6987,6988,6989,6990,6991,6992,6993,6994,3671,6995,6996,6997,6998,4839, // 6656
501 6999,7000,7001,7002,3549,7003,7004,7005,7006,7007,7008,7009,7010,7011,7012,7013, // 6672
502 7014,7015,7016,7017,7018,7019,7020,7021,7022,7023,7024,7025,7026,7027,7028,7029, // 6688
503 7030,4840,7031,7032,7033,7034,7035,7036,7037,7038,4841,7039,7040,7041,7042,7043, // 6704
504 7044,7045,7046,7047,7048,7049,7050,7051,7052,7053,7054,7055,7056,7057,7058,7059, // 6720
505 7060,7061,7062,7063,7064,7065,7066,7067,7068,7069,7070,2985,7071,7072,7073,7074, // 6736
506 7075,7076,7077,7078,7079,7080,4842,7081,7082,7083,7084,7085,7086,7087,7088,7089, // 6752
507 7090,7091,7092,7093,7094,7095,7096,7097,7098,7099,7100,7101,7102,7103,7104,7105, // 6768
508 7106,7107,7108,7109,7110,7111,7112,7113,7114,7115,7116,7117,7118,4450,7119,7120, // 6784
509 7121,7122,7123,7124,7125,7126,7127,7128,7129,7130,7131,7132,7133,7134,7135,7136, // 6800
510 7137,7138,7139,7140,7141,7142,7143,4843,7144,7145,7146,7147,7148,7149,7150,7151, // 6816
511 7152,7153,7154,7155,7156,7157,7158,7159,7160,7161,7162,7163,7164,7165,7166,7167, // 6832
512 7168,7169,7170,7171,7172,7173,7174,7175,7176,7177,7178,7179,7180,7181,7182,7183, // 6848
513 7184,7185,7186,7187,7188,4171,4172,7189,7190,7191,7192,7193,7194,7195,7196,7197, // 6864
514 7198,7199,7200,7201,7202,7203,7204,7205,7206,7207,7208,7209,7210,7211,7212,7213, // 6880
515 7214,7215,7216,7217,7218,7219,7220,7221,7222,7223,7224,7225,7226,7227,7228,7229, // 6896
516 7230,7231,7232,7233,7234,7235,7236,7237,7238,7239,7240,7241,7242,7243,7244,7245, // 6912
517 7246,7247,7248,7249,7250,7251,7252,7253,7254,7255,7256,7257,7258,7259,7260,7261, // 6928
518 7262,7263,7264,7265,7266,7267,7268,7269,7270,7271,7272,7273,7274,7275,7276,7277, // 6944
519 7278,7279,7280,7281,7282,7283,7284,7285,7286,7287,7288,7289,7290,7291,7292,7293, // 6960
520 7294,7295,7296,4844,7297,7298,7299,7300,7301,7302,7303,7304,7305,7306,7307,7308, // 6976
521 7309,7310,7311,7312,7313,7314,7315,7316,4451,7317,7318,7319,7320,7321,7322,7323, // 6992
522 7324,7325,7326,7327,7328,7329,7330,7331,7332,7333,7334,7335,7336,7337,7338,7339, // 7008
523 7340,7341,7342,7343,7344,7345,7346,7347,7348,7349,7350,7351,7352,7353,4173,7354, // 7024
524 7355,4845,7356,7357,7358,7359,7360,7361,7362,7363,7364,7365,7366,7367,7368,7369, // 7040
525 7370,7371,7372,7373,7374,7375,7376,7377,7378,7379,7380,7381,7382,7383,7384,7385, // 7056
526 7386,7387,7388,4846,7389,7390,7391,7392,7393,7394,7395,7396,7397,7398,7399,7400, // 7072
527 7401,7402,7403,7404,7405,3672,7406,7407,7408,7409,7410,7411,7412,7413,7414,7415, // 7088
528 7416,7417,7418,7419,7420,7421,7422,7423,7424,7425,7426,7427,7428,7429,7430,7431, // 7104
529 7432,7433,7434,7435,7436,7437,7438,7439,7440,7441,7442,7443,7444,7445,7446,7447, // 7120
530 7448,7449,7450,7451,7452,7453,4452,7454,3200,7455,7456,7457,7458,7459,7460,7461, // 7136
531 7462,7463,7464,7465,7466,7467,7468,7469,7470,7471,7472,7473,7474,4847,7475,7476, // 7152
532 7477,3133,7478,7479,7480,7481,7482,7483,7484,7485,7486,7487,7488,7489,7490,7491, // 7168
533 7492,7493,7494,7495,7496,7497,7498,7499,7500,7501,7502,3347,7503,7504,7505,7506, // 7184
534 7507,7508,7509,7510,7511,7512,7513,7514,7515,7516,7517,7518,7519,7520,7521,4848, // 7200
535 7522,7523,7524,7525,7526,7527,7528,7529,7530,7531,7532,7533,7534,7535,7536,7537, // 7216
536 7538,7539,7540,7541,7542,7543,7544,7545,7546,7547,7548,7549,3801,4849,7550,7551, // 7232
537 7552,7553,7554,7555,7556,7557,7558,7559,7560,7561,7562,7563,7564,7565,7566,7567, // 7248
538 7568,7569,3035,7570,7571,7572,7573,7574,7575,7576,7577,7578,7579,7580,7581,7582, // 7264
539 7583,7584,7585,7586,7587,7588,7589,7590,7591,7592,7593,7594,7595,7596,7597,7598, // 7280
540 7599,7600,7601,7602,7603,7604,7605,7606,7607,7608,7609,7610,7611,7612,7613,7614, // 7296
541 7615,7616,4850,7617,7618,3802,7619,7620,7621,7622,7623,7624,7625,7626,7627,7628, // 7312
542 7629,7630,7631,7632,4851,7633,7634,7635,7636,7637,7638,7639,7640,7641,7642,7643, // 7328
543 7644,7645,7646,7647,7648,7649,7650,7651,7652,7653,7654,7655,7656,7657,7658,7659, // 7344
544 7660,7661,7662,7663,7664,7665,7666,7667,7668,7669,7670,4453,7671,7672,7673,7674, // 7360
545 7675,7676,7677,7678,7679,7680,7681,7682,7683,7684,7685,7686,7687,7688,7689,7690, // 7376
546 7691,7692,7693,7694,7695,7696,7697,3443,7698,7699,7700,7701,7702,4454,7703,7704, // 7392
547 7705,7706,7707,7708,7709,7710,7711,7712,7713,2472,7714,7715,7716,7717,7718,7719, // 7408
548 7720,7721,7722,7723,7724,7725,7726,7727,7728,7729,7730,7731,3954,7732,7733,7734, // 7424
549 7735,7736,7737,7738,7739,7740,7741,7742,7743,7744,7745,7746,7747,7748,7749,7750, // 7440
550 3134,7751,7752,4852,7753,7754,7755,4853,7756,7757,7758,7759,7760,4174,7761,7762, // 7456
551 7763,7764,7765,7766,7767,7768,7769,7770,7771,7772,7773,7774,7775,7776,7777,7778, // 7472
552 7779,7780,7781,7782,7783,7784,7785,7786,7787,7788,7789,7790,7791,7792,7793,7794, // 7488
553 7795,7796,7797,7798,7799,7800,7801,7802,7803,7804,7805,4854,7806,7807,7808,7809, // 7504
554 7810,7811,7812,7813,7814,7815,7816,7817,7818,7819,7820,7821,7822,7823,7824,7825, // 7520
555 4855,7826,7827,7828,7829,7830,7831,7832,7833,7834,7835,7836,7837,7838,7839,7840, // 7536
556 7841,7842,7843,7844,7845,7846,7847,3955,7848,7849,7850,7851,7852,7853,7854,7855, // 7552
557 7856,7857,7858,7859,7860,3444,7861,7862,7863,7864,7865,7866,7867,7868,7869,7870, // 7568
558 7871,7872,7873,7874,7875,7876,7877,7878,7879,7880,7881,7882,7883,7884,7885,7886, // 7584
559 7887,7888,7889,7890,7891,4175,7892,7893,7894,7895,7896,4856,4857,7897,7898,7899, // 7600
560 7900,2598,7901,7902,7903,7904,7905,7906,7907,7908,4455,7909,7910,7911,7912,7913, // 7616
561 7914,3201,7915,7916,7917,7918,7919,7920,7921,4858,7922,7923,7924,7925,7926,7927, // 7632
562 7928,7929,7930,7931,7932,7933,7934,7935,7936,7937,7938,7939,7940,7941,7942,7943, // 7648
563 7944,7945,7946,7947,7948,7949,7950,7951,7952,7953,7954,7955,7956,7957,7958,7959, // 7664
564 7960,7961,7962,7963,7964,7965,7966,7967,7968,7969,7970,7971,7972,7973,7974,7975, // 7680
565 7976,7977,7978,7979,7980,7981,4859,7982,7983,7984,7985,7986,7987,7988,7989,7990, // 7696
566 7991,7992,7993,7994,7995,7996,4860,7997,7998,7999,8000,8001,8002,8003,8004,8005, // 7712
567 8006,8007,8008,8009,8010,8011,8012,8013,8014,8015,8016,4176,8017,8018,8019,8020, // 7728
568 8021,8022,8023,4861,8024,8025,8026,8027,8028,8029,8030,8031,8032,8033,8034,8035, // 7744
569 8036,4862,4456,8037,8038,8039,8040,4863,8041,8042,8043,8044,8045,8046,8047,8048, // 7760
570 8049,8050,8051,8052,8053,8054,8055,8056,8057,8058,8059,8060,8061,8062,8063,8064, // 7776
571 8065,8066,8067,8068,8069,8070,8071,8072,8073,8074,8075,8076,8077,8078,8079,8080, // 7792
572 8081,8082,8083,8084,8085,8086,8087,8088,8089,8090,8091,8092,8093,8094,8095,8096, // 7808
573 8097,8098,8099,4864,4177,8100,8101,8102,8103,8104,8105,8106,8107,8108,8109,8110, // 7824
574 8111,8112,8113,8114,8115,8116,8117,8118,8119,8120,4178,8121,8122,8123,8124,8125, // 7840
575 8126,8127,8128,8129,8130,8131,8132,8133,8134,8135,8136,8137,8138,8139,8140,8141, // 7856
576 8142,8143,8144,8145,4865,4866,8146,8147,8148,8149,8150,8151,8152,8153,8154,8155, // 7872
577 8156,8157,8158,8159,8160,8161,8162,8163,8164,8165,4179,8166,8167,8168,8169,8170, // 7888
578 8171,8172,8173,8174,8175,8176,8177,8178,8179,8180,8181,4457,8182,8183,8184,8185, // 7904
579 8186,8187,8188,8189,8190,8191,8192,8193,8194,8195,8196,8197,8198,8199,8200,8201, // 7920
580 8202,8203,8204,8205,8206,8207,8208,8209,8210,8211,8212,8213,8214,8215,8216,8217, // 7936
581 8218,8219,8220,8221,8222,8223,8224,8225,8226,8227,8228,8229,8230,8231,8232,8233, // 7952
582 8234,8235,8236,8237,8238,8239,8240,8241,8242,8243,8244,8245,8246,8247,8248,8249, // 7968
583 8250,8251,8252,8253,8254,8255,8256,3445,8257,8258,8259,8260,8261,8262,4458,8263, // 7984
584 8264,8265,8266,8267,8268,8269,8270,8271,8272,4459,8273,8274,8275,8276,3550,8277, // 8000
585 8278,8279,8280,8281,8282,8283,8284,8285,8286,8287,8288,8289,4460,8290,8291,8292, // 8016
586 8293,8294,8295,8296,8297,8298,8299,8300,8301,8302,8303,8304,8305,8306,8307,4867, // 8032
587 8308,8309,8310,8311,8312,3551,8313,8314,8315,8316,8317,8318,8319,8320,8321,8322, // 8048
588 8323,8324,8325,8326,4868,8327,8328,8329,8330,8331,8332,8333,8334,8335,8336,8337, // 8064
589 8338,8339,8340,8341,8342,8343,8344,8345,8346,8347,8348,8349,8350,8351,8352,8353, // 8080
590 8354,8355,8356,8357,8358,8359,8360,8361,8362,8363,4869,4461,8364,8365,8366,8367, // 8096
591 8368,8369,8370,4870,8371,8372,8373,8374,8375,8376,8377,8378,8379,8380,8381,8382, // 8112
592 8383,8384,8385,8386,8387,8388,8389,8390,8391,8392,8393,8394,8395,8396,8397,8398, // 8128
593 8399,8400,8401,8402,8403,8404,8405,8406,8407,8408,8409,8410,4871,8411,8412,8413, // 8144
594 8414,8415,8416,8417,8418,8419,8420,8421,8422,4462,8423,8424,8425,8426,8427,8428, // 8160
595 8429,8430,8431,8432,8433,2986,8434,8435,8436,8437,8438,8439,8440,8441,8442,8443, // 8176
596 8444,8445,8446,8447,8448,8449,8450,8451,8452,8453,8454,8455,8456,8457,8458,8459, // 8192
597 8460,8461,8462,8463,8464,8465,8466,8467,8468,8469,8470,8471,8472,8473,8474,8475, // 8208
598 8476,8477,8478,4180,8479,8480,8481,8482,8483,8484,8485,8486,8487,8488,8489,8490, // 8224
599 8491,8492,8493,8494,8495,8496,8497,8498,8499,8500,8501,8502,8503,8504,8505,8506, // 8240
600 8507,8508,8509,8510,8511,8512,8513,8514,8515,8516,8517,8518,8519,8520,8521,8522, // 8256
601 8523,8524,8525,8526,8527,8528,8529,8530,8531,8532,8533,8534,8535,8536,8537,8538, // 8272
602 8539,8540,8541,8542,8543,8544,8545,8546,8547,8548,8549,8550,8551,8552,8553,8554, // 8288
603 8555,8556,8557,8558,8559,8560,8561,8562,8563,8564,4872,8565,8566,8567,8568,8569, // 8304
604 8570,8571,8572,8573,4873,8574,8575,8576,8577,8578,8579,8580,8581,8582,8583,8584, // 8320
605 8585,8586,8587,8588,8589,8590,8591,8592,8593,8594,8595,8596,8597,8598,8599,8600, // 8336
606 8601,8602,8603,8604,8605,3803,8606,8607,8608,8609,8610,8611,8612,8613,4874,3804, // 8352
607 8614,8615,8616,8617,8618,8619,8620,8621,3956,8622,8623,8624,8625,8626,8627,8628, // 8368
608 8629,8630,8631,8632,8633,8634,8635,8636,8637,8638,2865,8639,8640,8641,8642,8643, // 8384
609 8644,8645,8646,8647,8648,8649,8650,8651,8652,8653,8654,8655,8656,4463,8657,8658, // 8400
610 8659,4875,4876,8660,8661,8662,8663,8664,8665,8666,8667,8668,8669,8670,8671,8672, // 8416
611 8673,8674,8675,8676,8677,8678,8679,8680,8681,4464,8682,8683,8684,8685,8686,8687, // 8432
612 8688,8689,8690,8691,8692,8693,8694,8695,8696,8697,8698,8699,8700,8701,8702,8703, // 8448
613 8704,8705,8706,8707,8708,8709,2261,8710,8711,8712,8713,8714,8715,8716,8717,8718, // 8464
614 8719,8720,8721,8722,8723,8724,8725,8726,8727,8728,8729,8730,8731,8732,8733,4181, // 8480
615 8734,8735,8736,8737,8738,8739,8740,8741,8742,8743,8744,8745,8746,8747,8748,8749, // 8496
616 8750,8751,8752,8753,8754,8755,8756,8757,8758,8759,8760,8761,8762,8763,4877,8764, // 8512
617 8765,8766,8767,8768,8769,8770,8771,8772,8773,8774,8775,8776,8777,8778,8779,8780, // 8528
618 8781,8782,8783,8784,8785,8786,8787,8788,4878,8789,4879,8790,8791,8792,4880,8793, // 8544
619 8794,8795,8796,8797,8798,8799,8800,8801,4881,8802,8803,8804,8805,8806,8807,8808, // 8560
620 8809,8810,8811,8812,8813,8814,8815,3957,8816,8817,8818,8819,8820,8821,8822,8823, // 8576
621 8824,8825,8826,8827,8828,8829,8830,8831,8832,8833,8834,8835,8836,8837,8838,8839, // 8592
622 8840,8841,8842,8843,8844,8845,8846,8847,4882,8848,8849,8850,8851,8852,8853,8854, // 8608
623 8855,8856,8857,8858,8859,8860,8861,8862,8863,8864,8865,8866,8867,8868,8869,8870, // 8624
624 8871,8872,8873,8874,8875,8876,8877,8878,8879,8880,8881,8882,8883,8884,3202,8885, // 8640
625 8886,8887,8888,8889,8890,8891,8892,8893,8894,8895,8896,8897,8898,8899,8900,8901, // 8656
626 8902,8903,8904,8905,8906,8907,8908,8909,8910,8911,8912,8913,8914,8915,8916,8917, // 8672
627 8918,8919,8920,8921,8922,8923,8924,4465,8925,8926,8927,8928,8929,8930,8931,8932, // 8688
628 4883,8933,8934,8935,8936,8937,8938,8939,8940,8941,8942,8943,2214,8944,8945,8946, // 8704
629 8947,8948,8949,8950,8951,8952,8953,8954,8955,8956,8957,8958,8959,8960,8961,8962, // 8720
630 8963,8964,8965,4884,8966,8967,8968,8969,8970,8971,8972,8973,8974,8975,8976,8977, // 8736
631 8978,8979,8980,8981,8982,8983,8984,8985,8986,8987,8988,8989,8990,8991,8992,4885, // 8752
632 8993,8994,8995,8996,8997,8998,8999,9000,9001,9002,9003,9004,9005,9006,9007,9008, // 8768
633 9009,9010,9011,9012,9013,9014,9015,9016,9017,9018,9019,9020,9021,4182,9022,9023, // 8784
634 9024,9025,9026,9027,9028,9029,9030,9031,9032,9033,9034,9035,9036,9037,9038,9039, // 8800
635 9040,9041,9042,9043,9044,9045,9046,9047,9048,9049,9050,9051,9052,9053,9054,9055, // 8816
636 9056,9057,9058,9059,9060,9061,9062,9063,4886,9064,9065,9066,9067,9068,9069,4887, // 8832
637 9070,9071,9072,9073,9074,9075,9076,9077,9078,9079,9080,9081,9082,9083,9084,9085, // 8848
638 9086,9087,9088,9089,9090,9091,9092,9093,9094,9095,9096,9097,9098,9099,9100,9101, // 8864
639 9102,9103,9104,9105,9106,9107,9108,9109,9110,9111,9112,9113,9114,9115,9116,9117, // 8880
640 9118,9119,9120,9121,9122,9123,9124,9125,9126,9127,9128,9129,9130,9131,9132,9133, // 8896
641 9134,9135,9136,9137,9138,9139,9140,9141,3958,9142,9143,9144,9145,9146,9147,9148, // 8912
642 9149,9150,9151,4888,9152,9153,9154,9155,9156,9157,9158,9159,9160,9161,9162,9163, // 8928
643 9164,9165,9166,9167,9168,9169,9170,9171,9172,9173,9174,9175,4889,9176,9177,9178, // 8944
644 9179,9180,9181,9182,9183,9184,9185,9186,9187,9188,9189,9190,9191,9192,9193,9194, // 8960
645 9195,9196,9197,9198,9199,9200,9201,9202,9203,4890,9204,9205,9206,9207,9208,9209, // 8976
646 9210,9211,9212,9213,9214,9215,9216,9217,9218,9219,9220,9221,9222,4466,9223,9224, // 8992
647 9225,9226,9227,9228,9229,9230,9231,9232,9233,9234,9235,9236,9237,9238,9239,9240, // 9008
648 9241,9242,9243,9244,9245,4891,9246,9247,9248,9249,9250,9251,9252,9253,9254,9255, // 9024
649 9256,9257,4892,9258,9259,9260,9261,4893,4894,9262,9263,9264,9265,9266,9267,9268, // 9040
650 9269,9270,9271,9272,9273,4467,9274,9275,9276,9277,9278,9279,9280,9281,9282,9283, // 9056
651 9284,9285,3673,9286,9287,9288,9289,9290,9291,9292,9293,9294,9295,9296,9297,9298, // 9072
652 9299,9300,9301,9302,9303,9304,9305,9306,9307,9308,9309,9310,9311,9312,9313,9314, // 9088
653 9315,9316,9317,9318,9319,9320,9321,9322,4895,9323,9324,9325,9326,9327,9328,9329, // 9104
654 9330,9331,9332,9333,9334,9335,9336,9337,9338,9339,9340,9341,9342,9343,9344,9345, // 9120
655 9346,9347,4468,9348,9349,9350,9351,9352,9353,9354,9355,9356,9357,9358,9359,9360, // 9136
656 9361,9362,9363,9364,9365,9366,9367,9368,9369,9370,9371,9372,9373,4896,9374,4469, // 9152
657 9375,9376,9377,9378,9379,4897,9380,9381,9382,9383,9384,9385,9386,9387,9388,9389, // 9168
658 9390,9391,9392,9393,9394,9395,9396,9397,9398,9399,9400,9401,9402,9403,9404,9405, // 9184
659 9406,4470,9407,2751,9408,9409,3674,3552,9410,9411,9412,9413,9414,9415,9416,9417, // 9200
660 9418,9419,9420,9421,4898,9422,9423,9424,9425,9426,9427,9428,9429,3959,9430,9431, // 9216
661 9432,9433,9434,9435,9436,4471,9437,9438,9439,9440,9441,9442,9443,9444,9445,9446, // 9232
662 9447,9448,9449,9450,3348,9451,9452,9453,9454,9455,9456,9457,9458,9459,9460,9461, // 9248
663 9462,9463,9464,9465,9466,9467,9468,9469,9470,9471,9472,4899,9473,9474,9475,9476, // 9264
664 9477,4900,9478,9479,9480,9481,9482,9483,9484,9485,9486,9487,9488,3349,9489,9490, // 9280
665 9491,9492,9493,9494,9495,9496,9497,9498,9499,9500,9501,9502,9503,9504,9505,9506, // 9296
666 9507,9508,9509,9510,9511,9512,9513,9514,9515,9516,9517,9518,9519,9520,4901,9521, // 9312
667 9522,9523,9524,9525,9526,4902,9527,9528,9529,9530,9531,9532,9533,9534,9535,9536, // 9328
668 9537,9538,9539,9540,9541,9542,9543,9544,9545,9546,9547,9548,9549,9550,9551,9552, // 9344
669 9553,9554,9555,9556,9557,9558,9559,9560,9561,9562,9563,9564,9565,9566,9567,9568, // 9360
670 9569,9570,9571,9572,9573,9574,9575,9576,9577,9578,9579,9580,9581,9582,9583,9584, // 9376
671 3805,9585,9586,9587,9588,9589,9590,9591,9592,9593,9594,9595,9596,9597,9598,9599, // 9392
672 9600,9601,9602,4903,9603,9604,9605,9606,9607,4904,9608,9609,9610,9611,9612,9613, // 9408
673 9614,4905,9615,9616,9617,9618,9619,9620,9621,9622,9623,9624,9625,9626,9627,9628, // 9424
674 9629,9630,9631,9632,4906,9633,9634,9635,9636,9637,9638,9639,9640,9641,9642,9643, // 9440
675 4907,9644,9645,9646,9647,9648,9649,9650,9651,9652,9653,9654,9655,9656,9657,9658, // 9456
676 9659,9660,9661,9662,9663,9664,9665,9666,9667,9668,9669,9670,9671,9672,4183,9673, // 9472
677 9674,9675,9676,9677,4908,9678,9679,9680,9681,4909,9682,9683,9684,9685,9686,9687, // 9488
678 9688,9689,9690,4910,9691,9692,9693,3675,9694,9695,9696,2945,9697,9698,9699,9700, // 9504
679 9701,9702,9703,9704,9705,4911,9706,9707,9708,9709,9710,9711,9712,9713,9714,9715, // 9520
680 9716,9717,9718,9719,9720,9721,9722,9723,9724,9725,9726,9727,9728,9729,9730,9731, // 9536
681 9732,9733,9734,9735,4912,9736,9737,9738,9739,9740,4913,9741,9742,9743,9744,9745, // 9552
682 9746,9747,9748,9749,9750,9751,9752,9753,9754,9755,9756,9757,9758,4914,9759,9760, // 9568
683 9761,9762,9763,9764,9765,9766,9767,9768,9769,9770,9771,9772,9773,9774,9775,9776, // 9584
684 9777,9778,9779,9780,9781,9782,4915,9783,9784,9785,9786,9787,9788,9789,9790,9791, // 9600
685 9792,9793,4916,9794,9795,9796,9797,9798,9799,9800,9801,9802,9803,9804,9805,9806, // 9616
686 9807,9808,9809,9810,9811,9812,9813,9814,9815,9816,9817,9818,9819,9820,9821,9822, // 9632
687 9823,9824,9825,9826,9827,9828,9829,9830,9831,9832,9833,9834,9835,9836,9837,9838, // 9648
688 9839,9840,9841,9842,9843,9844,9845,9846,9847,9848,9849,9850,9851,9852,9853,9854, // 9664
689 9855,9856,9857,9858,9859,9860,9861,9862,9863,9864,9865,9866,9867,9868,4917,9869, // 9680
690 9870,9871,9872,9873,9874,9875,9876,9877,9878,9879,9880,9881,9882,9883,9884,9885, // 9696
691 9886,9887,9888,9889,9890,9891,9892,4472,9893,9894,9895,9896,9897,3806,9898,9899, // 9712
692 9900,9901,9902,9903,9904,9905,9906,9907,9908,9909,9910,9911,9912,9913,9914,4918, // 9728
693 9915,9916,9917,4919,9918,9919,9920,9921,4184,9922,9923,9924,9925,9926,9927,9928, // 9744
694 9929,9930,9931,9932,9933,9934,9935,9936,9937,9938,9939,9940,9941,9942,9943,9944, // 9760
695 9945,9946,4920,9947,9948,9949,9950,9951,9952,9953,9954,9955,4185,9956,9957,9958, // 9776
696 9959,9960,9961,9962,9963,9964,9965,4921,9966,9967,9968,4473,9969,9970,9971,9972, // 9792
697 9973,9974,9975,9976,9977,4474,9978,9979,9980,9981,9982,9983,9984,9985,9986,9987, // 9808
698 9988,9989,9990,9991,9992,9993,9994,9995,9996,9997,9998,9999,10000,10001,10002,10003, // 9824
699 10004,10005,10006,10007,10008,10009,10010,10011,10012,10013,10014,10015,10016,10017,10018,10019, // 9840
700 10020,10021,4922,10022,4923,10023,10024,10025,10026,10027,10028,10029,10030,10031,10032,10033, // 9856
701 10034,10035,10036,10037,10038,10039,10040,10041,10042,10043,10044,10045,10046,10047,10048,4924, // 9872
702 10049,10050,10051,10052,10053,10054,10055,10056,10057,10058,10059,10060,10061,10062,10063,10064, // 9888
703 10065,10066,10067,10068,10069,10070,10071,10072,10073,10074,10075,10076,10077,10078,10079,10080, // 9904
704 10081,10082,10083,10084,10085,10086,10087,4475,10088,10089,10090,10091,10092,10093,10094,10095, // 9920
705 10096,10097,4476,10098,10099,10100,10101,10102,10103,10104,10105,10106,10107,10108,10109,10110, // 9936
706 10111,2174,10112,10113,10114,10115,10116,10117,10118,10119,10120,10121,10122,10123,10124,10125, // 9952
707 10126,10127,10128,10129,10130,10131,10132,10133,10134,10135,10136,10137,10138,10139,10140,3807, // 9968
708 4186,4925,10141,10142,10143,10144,10145,10146,10147,4477,4187,10148,10149,10150,10151,10152, // 9984
709 10153,4188,10154,10155,10156,10157,10158,10159,10160,10161,4926,10162,10163,10164,10165,10166, //10000
710 10167,10168,10169,10170,10171,10172,10173,10174,10175,10176,10177,10178,10179,10180,10181,10182, //10016
711 10183,10184,10185,10186,10187,10188,10189,10190,10191,10192,3203,10193,10194,10195,10196,10197, //10032
712 10198,10199,10200,4478,10201,10202,10203,10204,4479,10205,10206,10207,10208,10209,10210,10211, //10048
713 10212,10213,10214,10215,10216,10217,10218,10219,10220,10221,10222,10223,10224,10225,10226,10227, //10064
714 10228,10229,10230,10231,10232,10233,10234,4927,10235,10236,10237,10238,10239,10240,10241,10242, //10080
715 10243,10244,10245,10246,10247,10248,10249,10250,10251,10252,10253,10254,10255,10256,10257,10258, //10096
716 10259,10260,10261,10262,10263,10264,10265,10266,10267,10268,10269,10270,10271,10272,10273,4480, //10112
717 4928,4929,10274,10275,10276,10277,10278,10279,10280,10281,10282,10283,10284,10285,10286,10287, //10128
718 10288,10289,10290,10291,10292,10293,10294,10295,10296,10297,10298,10299,10300,10301,10302,10303, //10144
719 10304,10305,10306,10307,10308,10309,10310,10311,10312,10313,10314,10315,10316,10317,10318,10319, //10160
720 10320,10321,10322,10323,10324,10325,10326,10327,10328,10329,10330,10331,10332,10333,10334,4930, //10176
721 10335,10336,10337,10338,10339,10340,10341,10342,4931,10343,10344,10345,10346,10347,10348,10349, //10192
722 10350,10351,10352,10353,10354,10355,3088,10356,2786,10357,10358,10359,10360,4189,10361,10362, //10208
723 10363,10364,10365,10366,10367,10368,10369,10370,10371,10372,10373,10374,10375,4932,10376,10377, //10224
724 10378,10379,10380,10381,10382,10383,10384,10385,10386,10387,10388,10389,10390,10391,10392,4933, //10240
725 10393,10394,10395,4934,10396,10397,10398,10399,10400,10401,10402,10403,10404,10405,10406,10407, //10256
726 10408,10409,10410,10411,10412,3446,10413,10414,10415,10416,10417,10418,10419,10420,10421,10422, //10272
727 10423,4935,10424,10425,10426,10427,10428,10429,10430,4936,10431,10432,10433,10434,10435,10436, //10288
728 10437,10438,10439,10440,10441,10442,10443,4937,10444,10445,10446,10447,4481,10448,10449,10450, //10304
729 10451,10452,10453,10454,10455,10456,10457,10458,10459,10460,10461,10462,10463,10464,10465,10466, //10320
730 10467,10468,10469,10470,10471,10472,10473,10474,10475,10476,10477,10478,10479,10480,10481,10482, //10336
731 10483,10484,10485,10486,10487,10488,10489,10490,10491,10492,10493,10494,10495,10496,10497,10498, //10352
732 10499,10500,10501,10502,10503,10504,10505,4938,10506,10507,10508,10509,10510,2552,10511,10512, //10368
733 10513,10514,10515,10516,3447,10517,10518,10519,10520,10521,10522,10523,10524,10525,10526,10527, //10384
734 10528,10529,10530,10531,10532,10533,10534,10535,10536,10537,10538,10539,10540,10541,10542,10543, //10400
735 4482,10544,4939,10545,10546,10547,10548,10549,10550,10551,10552,10553,10554,10555,10556,10557, //10416
736 10558,10559,10560,10561,10562,10563,10564,10565,10566,10567,3676,4483,10568,10569,10570,10571, //10432
737 10572,3448,10573,10574,10575,10576,10577,10578,10579,10580,10581,10582,10583,10584,10585,10586, //10448
738 10587,10588,10589,10590,10591,10592,10593,10594,10595,10596,10597,10598,10599,10600,10601,10602, //10464
739 10603,10604,10605,10606,10607,10608,10609,10610,10611,10612,10613,10614,10615,10616,10617,10618, //10480
740 10619,10620,10621,10622,10623,10624,10625,10626,10627,4484,10628,10629,10630,10631,10632,4940, //10496
741 10633,10634,10635,10636,10637,10638,10639,10640,10641,10642,10643,10644,10645,10646,10647,10648, //10512
742 10649,10650,10651,10652,10653,10654,10655,10656,4941,10657,10658,10659,2599,10660,10661,10662, //10528
743 10663,10664,10665,10666,3089,10667,10668,10669,10670,10671,10672,10673,10674,10675,10676,10677, //10544
744 10678,10679,10680,4942,10681,10682,10683,10684,10685,10686,10687,10688,10689,10690,10691,10692, //10560
745 10693,10694,10695,10696,10697,4485,10698,10699,10700,10701,10702,10703,10704,4943,10705,3677, //10576
746 10706,10707,10708,10709,10710,10711,10712,4944,10713,10714,10715,10716,10717,10718,10719,10720, //10592
747 10721,10722,10723,10724,10725,10726,10727,10728,4945,10729,10730,10731,10732,10733,10734,10735, //10608
748 10736,10737,10738,10739,10740,10741,10742,10743,10744,10745,10746,10747,10748,10749,10750,10751, //10624
749 10752,10753,10754,10755,10756,10757,10758,10759,10760,10761,4946,10762,10763,10764,10765,10766, //10640
750 10767,4947,4948,10768,10769,10770,10771,10772,10773,10774,10775,10776,10777,10778,10779,10780, //10656
751 10781,10782,10783,10784,10785,10786,10787,10788,10789,10790,10791,10792,10793,10794,10795,10796, //10672
752 10797,10798,10799,10800,10801,10802,10803,10804,10805,10806,10807,10808,10809,10810,10811,10812, //10688
753 10813,10814,10815,10816,10817,10818,10819,10820,10821,10822,10823,10824,10825,10826,10827,10828, //10704
754 10829,10830,10831,10832,10833,10834,10835,10836,10837,10838,10839,10840,10841,10842,10843,10844, //10720
755 10845,10846,10847,10848,10849,10850,10851,10852,10853,10854,10855,10856,10857,10858,10859,10860, //10736
756 10861,10862,10863,10864,10865,10866,10867,10868,10869,10870,10871,10872,10873,10874,10875,10876, //10752
757 10877,10878,4486,10879,10880,10881,10882,10883,10884,10885,4949,10886,10887,10888,10889,10890, //10768
758 10891,10892,10893,10894,10895,10896,10897,10898,10899,10900,10901,10902,10903,10904,10905,10906, //10784
759 10907,10908,10909,10910,10911,10912,10913,10914,10915,10916,10917,10918,10919,4487,10920,10921, //10800
760 10922,10923,10924,10925,10926,10927,10928,10929,10930,10931,10932,4950,10933,10934,10935,10936, //10816
761 10937,10938,10939,10940,10941,10942,10943,10944,10945,10946,10947,10948,10949,4488,10950,10951, //10832
762 10952,10953,10954,10955,10956,10957,10958,10959,4190,10960,10961,10962,10963,10964,10965,10966, //10848
763 10967,10968,10969,10970,10971,10972,10973,10974,10975,10976,10977,10978,10979,10980,10981,10982, //10864
764 10983,10984,10985,10986,10987,10988,10989,10990,10991,10992,10993,10994,10995,10996,10997,10998, //10880
765 10999,11000,11001,11002,11003,11004,11005,11006,3960,11007,11008,11009,11010,11011,11012,11013, //10896
766 11014,11015,11016,11017,11018,11019,11020,11021,11022,11023,11024,11025,11026,11027,11028,11029, //10912
767 11030,11031,11032,4951,11033,11034,11035,11036,11037,11038,11039,11040,11041,11042,11043,11044, //10928
768 11045,11046,11047,4489,11048,11049,11050,11051,4952,11052,11053,11054,11055,11056,11057,11058, //10944
769 4953,11059,11060,11061,11062,11063,11064,11065,11066,11067,11068,11069,11070,11071,4954,11072, //10960
770 11073,11074,11075,11076,11077,11078,11079,11080,11081,11082,11083,11084,11085,11086,11087,11088, //10976
771 11089,11090,11091,11092,11093,11094,11095,11096,11097,11098,11099,11100,11101,11102,11103,11104, //10992
772 11105,11106,11107,11108,11109,11110,11111,11112,11113,11114,11115,3808,11116,11117,11118,11119, //11008
773 11120,11121,11122,11123,11124,11125,11126,11127,11128,11129,11130,11131,11132,11133,11134,4955, //11024
774 11135,11136,11137,11138,11139,11140,11141,11142,11143,11144,11145,11146,11147,11148,11149,11150, //11040
775 11151,11152,11153,11154,11155,11156,11157,11158,11159,11160,11161,4956,11162,11163,11164,11165, //11056
776 11166,11167,11168,11169,11170,11171,11172,11173,11174,11175,11176,11177,11178,11179,11180,4957, //11072
777 11181,11182,11183,11184,11185,11186,4958,11187,11188,11189,11190,11191,11192,11193,11194,11195, //11088
778 11196,11197,11198,11199,11200,3678,11201,11202,11203,11204,11205,11206,4191,11207,11208,11209, //11104
779 11210,11211,11212,11213,11214,11215,11216,11217,11218,11219,11220,11221,11222,11223,11224,11225, //11120
780 11226,11227,11228,11229,11230,11231,11232,11233,11234,11235,11236,11237,11238,11239,11240,11241, //11136
781 11242,11243,11244,11245,11246,11247,11248,11249,11250,11251,4959,11252,11253,11254,11255,11256, //11152
782 11257,11258,11259,11260,11261,11262,11263,11264,11265,11266,11267,11268,11269,11270,11271,11272, //11168
783 11273,11274,11275,11276,11277,11278,11279,11280,11281,11282,11283,11284,11285,11286,11287,11288, //11184
784 11289,11290,11291,11292,11293,11294,11295,11296,11297,11298,11299,11300,11301,11302,11303,11304, //11200
785 11305,11306,11307,11308,11309,11310,11311,11312,11313,11314,3679,11315,11316,11317,11318,4490, //11216
786 11319,11320,11321,11322,11323,11324,11325,11326,11327,11328,11329,11330,11331,11332,11333,11334, //11232
787 11335,11336,11337,11338,11339,11340,11341,11342,11343,11344,11345,11346,11347,4960,11348,11349, //11248
788 11350,11351,11352,11353,11354,11355,11356,11357,11358,11359,11360,11361,11362,11363,11364,11365, //11264
789 11366,11367,11368,11369,11370,11371,11372,11373,11374,11375,11376,11377,3961,4961,11378,11379, //11280
790 11380,11381,11382,11383,11384,11385,11386,11387,11388,11389,11390,11391,11392,11393,11394,11395, //11296
791 11396,11397,4192,11398,11399,11400,11401,11402,11403,11404,11405,11406,11407,11408,11409,11410, //11312
792 11411,4962,11412,11413,11414,11415,11416,11417,11418,11419,11420,11421,11422,11423,11424,11425, //11328
793 11426,11427,11428,11429,11430,11431,11432,11433,11434,11435,11436,11437,11438,11439,11440,11441, //11344
794 11442,11443,11444,11445,11446,11447,11448,11449,11450,11451,11452,11453,11454,11455,11456,11457, //11360
795 11458,11459,11460,11461,11462,11463,11464,11465,11466,11467,11468,11469,4963,11470,11471,4491, //11376
796 11472,11473,11474,11475,4964,11476,11477,11478,11479,11480,11481,11482,11483,11484,11485,11486, //11392
797 11487,11488,11489,11490,11491,11492,4965,11493,11494,11495,11496,11497,11498,11499,11500,11501, //11408
798 11502,11503,11504,11505,11506,11507,11508,11509,11510,11511,11512,11513,11514,11515,11516,11517, //11424
799 11518,11519,11520,11521,11522,11523,11524,11525,11526,11527,11528,11529,3962,11530,11531,11532, //11440
800 11533,11534,11535,11536,11537,11538,11539,11540,11541,11542,11543,11544,11545,11546,11547,11548, //11456
801 11549,11550,11551,11552,11553,11554,11555,11556,11557,11558,11559,11560,11561,11562,11563,11564, //11472
802 4193,4194,11565,11566,11567,11568,11569,11570,11571,11572,11573,11574,11575,11576,11577,11578, //11488
803 11579,11580,11581,11582,11583,11584,11585,11586,11587,11588,11589,11590,11591,4966,4195,11592, //11504
804 11593,11594,11595,11596,11597,11598,11599,11600,11601,11602,11603,11604,3090,11605,11606,11607, //11520
805 11608,11609,11610,4967,11611,11612,11613,11614,11615,11616,11617,11618,11619,11620,11621,11622, //11536
806 11623,11624,11625,11626,11627,11628,11629,11630,11631,11632,11633,11634,11635,11636,11637,11638, //11552
807 11639,11640,11641,11642,11643,11644,11645,11646,11647,11648,11649,11650,11651,11652,11653,11654, //11568
808 11655,11656,11657,11658,11659,11660,11661,11662,11663,11664,11665,11666,11667,11668,11669,11670, //11584
809 11671,11672,11673,11674,4968,11675,11676,11677,11678,11679,11680,11681,11682,11683,11684,11685, //11600
810 11686,11687,11688,11689,11690,11691,11692,11693,3809,11694,11695,11696,11697,11698,11699,11700, //11616
811 11701,11702,11703,11704,11705,11706,11707,11708,11709,11710,11711,11712,11713,11714,11715,11716, //11632
812 11717,11718,3553,11719,11720,11721,11722,11723,11724,11725,11726,11727,11728,11729,11730,4969, //11648
813 11731,11732,11733,11734,11735,11736,11737,11738,11739,11740,4492,11741,11742,11743,11744,11745, //11664
814 11746,11747,11748,11749,11750,11751,11752,4970,11753,11754,11755,11756,11757,11758,11759,11760, //11680
815 11761,11762,11763,11764,11765,11766,11767,11768,11769,11770,11771,11772,11773,11774,11775,11776, //11696
816 11777,11778,11779,11780,11781,11782,11783,11784,11785,11786,11787,11788,11789,11790,4971,11791, //11712
817 11792,11793,11794,11795,11796,11797,4972,11798,11799,11800,11801,11802,11803,11804,11805,11806, //11728
818 11807,11808,11809,11810,4973,11811,11812,11813,11814,11815,11816,11817,11818,11819,11820,11821, //11744
819 11822,11823,11824,11825,11826,11827,11828,11829,11830,11831,11832,11833,11834,3680,3810,11835, //11760
820 11836,4974,11837,11838,11839,11840,11841,11842,11843,11844,11845,11846,11847,11848,11849,11850, //11776
821 11851,11852,11853,11854,11855,11856,11857,11858,11859,11860,11861,11862,11863,11864,11865,11866, //11792
822 11867,11868,11869,11870,11871,11872,11873,11874,11875,11876,11877,11878,11879,11880,11881,11882, //11808
823 11883,11884,4493,11885,11886,11887,11888,11889,11890,11891,11892,11893,11894,11895,11896,11897, //11824
824 11898,11899,11900,11901,11902,11903,11904,11905,11906,11907,11908,11909,11910,11911,11912,11913, //11840
825 11914,11915,4975,11916,11917,11918,11919,11920,11921,11922,11923,11924,11925,11926,11927,11928, //11856
826 11929,11930,11931,11932,11933,11934,11935,11936,11937,11938,11939,11940,11941,11942,11943,11944, //11872
827 11945,11946,11947,11948,11949,4976,11950,11951,11952,11953,11954,11955,11956,11957,11958,11959, //11888
828 11960,11961,11962,11963,11964,11965,11966,11967,11968,11969,11970,11971,11972,11973,11974,11975, //11904
829 11976,11977,11978,11979,11980,11981,11982,11983,11984,11985,11986,11987,4196,11988,11989,11990, //11920
830 11991,11992,4977,11993,11994,11995,11996,11997,11998,11999,12000,12001,12002,12003,12004,12005, //11936
831 12006,12007,12008,12009,12010,12011,12012,12013,12014,12015,12016,12017,12018,12019,12020,12021, //11952
832 12022,12023,12024,12025,12026,12027,12028,12029,12030,12031,12032,12033,12034,12035,12036,12037, //11968
833 12038,12039,12040,12041,12042,12043,12044,12045,12046,12047,12048,12049,12050,12051,12052,12053, //11984
834 12054,12055,12056,12057,12058,12059,12060,12061,4978,12062,12063,12064,12065,12066,12067,12068, //12000
835 12069,12070,12071,12072,12073,12074,12075,12076,12077,12078,12079,12080,12081,12082,12083,12084, //12016
836 12085,12086,12087,12088,12089,12090,12091,12092,12093,12094,12095,12096,12097,12098,12099,12100, //12032
837 12101,12102,12103,12104,12105,12106,12107,12108,12109,12110,12111,12112,12113,12114,12115,12116, //12048
838 12117,12118,12119,12120,12121,12122,12123,4979,12124,12125,12126,12127,12128,4197,12129,12130, //12064
839 12131,12132,12133,12134,12135,12136,12137,12138,12139,12140,12141,12142,12143,12144,12145,12146, //12080
840 12147,12148,12149,12150,12151,12152,12153,12154,4980,12155,12156,12157,12158,12159,12160,4494, //12096
841 12161,12162,12163,12164,3811,12165,12166,12167,12168,12169,4495,12170,12171,4496,12172,12173, //12112
842 12174,12175,12176,3812,12177,12178,12179,12180,12181,12182,12183,12184,12185,12186,12187,12188, //12128
843 12189,12190,12191,12192,12193,12194,12195,12196,12197,12198,12199,12200,12201,12202,12203,12204, //12144
844 12205,12206,12207,12208,12209,12210,12211,12212,12213,12214,12215,12216,12217,12218,12219,12220, //12160
845 12221,4981,12222,12223,12224,12225,12226,12227,12228,12229,12230,12231,12232,12233,12234,12235, //12176
846 4982,12236,12237,12238,12239,12240,12241,12242,12243,12244,12245,4983,12246,12247,12248,12249, //12192
847 4984,12250,12251,12252,12253,12254,12255,12256,12257,12258,12259,12260,12261,12262,12263,12264, //12208
848 4985,12265,4497,12266,12267,12268,12269,12270,12271,12272,12273,12274,12275,12276,12277,12278, //12224
849 12279,12280,12281,12282,12283,12284,12285,12286,12287,4986,12288,12289,12290,12291,12292,12293, //12240
850 12294,12295,12296,2473,12297,12298,12299,12300,12301,12302,12303,12304,12305,12306,12307,12308, //12256
851 12309,12310,12311,12312,12313,12314,12315,12316,12317,12318,12319,3963,12320,12321,12322,12323, //12272
852 12324,12325,12326,12327,12328,12329,12330,12331,12332,4987,12333,12334,12335,12336,12337,12338, //12288
853 12339,12340,12341,12342,12343,12344,12345,12346,12347,12348,12349,12350,12351,12352,12353,12354, //12304
854 12355,12356,12357,12358,12359,3964,12360,12361,12362,12363,12364,12365,12366,12367,12368,12369, //12320
855 12370,3965,12371,12372,12373,12374,12375,12376,12377,12378,12379,12380,12381,12382,12383,12384, //12336
856 12385,12386,12387,12388,12389,12390,12391,12392,12393,12394,12395,12396,12397,12398,12399,12400, //12352
857 12401,12402,12403,12404,12405,12406,12407,12408,4988,12409,12410,12411,12412,12413,12414,12415, //12368
858 12416,12417,12418,12419,12420,12421,12422,12423,12424,12425,12426,12427,12428,12429,12430,12431, //12384
859 12432,12433,12434,12435,12436,12437,12438,3554,12439,12440,12441,12442,12443,12444,12445,12446, //12400
860 12447,12448,12449,12450,12451,12452,12453,12454,12455,12456,12457,12458,12459,12460,12461,12462, //12416
861 12463,12464,4989,12465,12466,12467,12468,12469,12470,12471,12472,12473,12474,12475,12476,12477, //12432
862 12478,12479,12480,4990,12481,12482,12483,12484,12485,12486,12487,12488,12489,4498,12490,12491, //12448
863 12492,12493,12494,12495,12496,12497,12498,12499,12500,12501,12502,12503,12504,12505,12506,12507, //12464
864 12508,12509,12510,12511,12512,12513,12514,12515,12516,12517,12518,12519,12520,12521,12522,12523, //12480
865 12524,12525,12526,12527,12528,12529,12530,12531,12532,12533,12534,12535,12536,12537,12538,12539, //12496
866 12540,12541,12542,12543,12544,12545,12546,12547,12548,12549,12550,12551,4991,12552,12553,12554, //12512
867 12555,12556,12557,12558,12559,12560,12561,12562,12563,12564,12565,12566,12567,12568,12569,12570, //12528
868 12571,12572,12573,12574,12575,12576,12577,12578,3036,12579,12580,12581,12582,12583,3966,12584, //12544
869 12585,12586,12587,12588,12589,12590,12591,12592,12593,12594,12595,12596,12597,12598,12599,12600, //12560
870 12601,12602,12603,12604,12605,12606,12607,12608,12609,12610,12611,12612,12613,12614,12615,12616, //12576
871 12617,12618,12619,12620,12621,12622,12623,12624,12625,12626,12627,12628,12629,12630,12631,12632, //12592
872 12633,12634,12635,12636,12637,12638,12639,12640,12641,12642,12643,12644,12645,12646,4499,12647, //12608
873 12648,12649,12650,12651,12652,12653,12654,12655,12656,12657,12658,12659,12660,12661,12662,12663, //12624
874 12664,12665,12666,12667,12668,12669,12670,12671,12672,12673,12674,12675,12676,12677,12678,12679, //12640
875 12680,12681,12682,12683,12684,12685,12686,12687,12688,12689,12690,12691,12692,12693,12694,12695, //12656
876 12696,12697,12698,4992,12699,12700,12701,12702,12703,12704,12705,12706,12707,12708,12709,12710, //12672
877 12711,12712,12713,12714,12715,12716,12717,12718,12719,12720,12721,12722,12723,12724,12725,12726, //12688
878 12727,12728,12729,12730,12731,12732,12733,12734,12735,12736,12737,12738,12739,12740,12741,12742, //12704
879 12743,12744,12745,12746,12747,12748,12749,12750,12751,12752,12753,12754,12755,12756,12757,12758, //12720
880 12759,12760,12761,12762,12763,12764,12765,12766,12767,12768,12769,12770,12771,12772,12773,12774, //12736
881 12775,12776,12777,12778,4993,2175,12779,12780,12781,12782,12783,12784,12785,12786,4500,12787, //12752
882 12788,12789,12790,12791,12792,12793,12794,12795,12796,12797,12798,12799,12800,12801,12802,12803, //12768
883 12804,12805,12806,12807,12808,12809,12810,12811,12812,12813,12814,12815,12816,12817,12818,12819, //12784
884 12820,12821,12822,12823,12824,12825,12826,4198,3967,12827,12828,12829,12830,12831,12832,12833, //12800
885 12834,12835,12836,12837,12838,12839,12840,12841,12842,12843,12844,12845,12846,12847,12848,12849, //12816
886 12850,12851,12852,12853,12854,12855,12856,12857,12858,12859,12860,12861,4199,12862,12863,12864, //12832
887 12865,12866,12867,12868,12869,12870,12871,12872,12873,12874,12875,12876,12877,12878,12879,12880, //12848
888 12881,12882,12883,12884,12885,12886,12887,4501,12888,12889,12890,12891,12892,12893,12894,12895, //12864
889 12896,12897,12898,12899,12900,12901,12902,12903,12904,12905,12906,12907,12908,12909,12910,12911, //12880
890 12912,4994,12913,12914,12915,12916,12917,12918,12919,12920,12921,12922,12923,12924,12925,12926, //12896
891 12927,12928,12929,12930,12931,12932,12933,12934,12935,12936,12937,12938,12939,12940,12941,12942, //12912
892 12943,12944,12945,12946,12947,12948,12949,12950,12951,12952,12953,12954,12955,12956,1772,12957, //12928
893 12958,12959,12960,12961,12962,12963,12964,12965,12966,12967,12968,12969,12970,12971,12972,12973, //12944
894 12974,12975,12976,12977,12978,12979,12980,12981,12982,12983,12984,12985,12986,12987,12988,12989, //12960
895 12990,12991,12992,12993,12994,12995,12996,12997,4502,12998,4503,12999,13000,13001,13002,13003, //12976
896 4504,13004,13005,13006,13007,13008,13009,13010,13011,13012,13013,13014,13015,13016,13017,13018, //12992
897 13019,13020,13021,13022,13023,13024,13025,13026,13027,13028,13029,3449,13030,13031,13032,13033, //13008
898 13034,13035,13036,13037,13038,13039,13040,13041,13042,13043,13044,13045,13046,13047,13048,13049, //13024
899 13050,13051,13052,13053,13054,13055,13056,13057,13058,13059,13060,13061,13062,13063,13064,13065, //13040
900 13066,13067,13068,13069,13070,13071,13072,13073,13074,13075,13076,13077,13078,13079,13080,13081, //13056
901 13082,13083,13084,13085,13086,13087,13088,13089,13090,13091,13092,13093,13094,13095,13096,13097, //13072
902 13098,13099,13100,13101,13102,13103,13104,13105,13106,13107,13108,13109,13110,13111,13112,13113, //13088
903 13114,13115,13116,13117,13118,3968,13119,4995,13120,13121,13122,13123,13124,13125,13126,13127, //13104
904 4505,13128,13129,13130,13131,13132,13133,13134,4996,4506,13135,13136,13137,13138,13139,4997, //13120
905 13140,13141,13142,13143,13144,13145,13146,13147,13148,13149,13150,13151,13152,13153,13154,13155, //13136
906 13156,13157,13158,13159,4998,13160,13161,13162,13163,13164,13165,13166,13167,13168,13169,13170, //13152
907 13171,13172,13173,13174,13175,13176,4999,13177,13178,13179,13180,13181,13182,13183,13184,13185, //13168
908 13186,13187,13188,13189,13190,13191,13192,13193,13194,13195,13196,13197,13198,13199,13200,13201, //13184
909 13202,13203,13204,13205,13206,5000,13207,13208,13209,13210,13211,13212,13213,13214,13215,13216, //13200
910 13217,13218,13219,13220,13221,13222,13223,13224,13225,13226,13227,4200,5001,13228,13229,13230, //13216
911 13231,13232,13233,13234,13235,13236,13237,13238,13239,13240,3969,13241,13242,13243,13244,3970, //13232
912 13245,13246,13247,13248,13249,13250,13251,13252,13253,13254,13255,13256,13257,13258,13259,13260, //13248
913 13261,13262,13263,13264,13265,13266,13267,13268,3450,13269,13270,13271,13272,13273,13274,13275, //13264
914 13276,5002,13277,13278,13279,13280,13281,13282,13283,13284,13285,13286,13287,13288,13289,13290, //13280
915 13291,13292,13293,13294,13295,13296,13297,13298,13299,13300,13301,13302,3813,13303,13304,13305, //13296
916 13306,13307,13308,13309,13310,13311,13312,13313,13314,13315,13316,13317,13318,13319,13320,13321, //13312
917 13322,13323,13324,13325,13326,13327,13328,4507,13329,13330,13331,13332,13333,13334,13335,13336, //13328
918 13337,13338,13339,13340,13341,5003,13342,13343,13344,13345,13346,13347,13348,13349,13350,13351, //13344
919 13352,13353,13354,13355,13356,13357,13358,13359,13360,13361,13362,13363,13364,13365,13366,13367, //13360
920 5004,13368,13369,13370,13371,13372,13373,13374,13375,13376,13377,13378,13379,13380,13381,13382, //13376
921 13383,13384,13385,13386,13387,13388,13389,13390,13391,13392,13393,13394,13395,13396,13397,13398, //13392
922 13399,13400,13401,13402,13403,13404,13405,13406,13407,13408,13409,13410,13411,13412,13413,13414, //13408
923 13415,13416,13417,13418,13419,13420,13421,13422,13423,13424,13425,13426,13427,13428,13429,13430, //13424
924 13431,13432,4508,13433,13434,13435,4201,13436,13437,13438,13439,13440,13441,13442,13443,13444, //13440
925 13445,13446,13447,13448,13449,13450,13451,13452,13453,13454,13455,13456,13457,5005,13458,13459, //13456
926 13460,13461,13462,13463,13464,13465,13466,13467,13468,13469,13470,4509,13471,13472,13473,13474, //13472
927 13475,13476,13477,13478,13479,13480,13481,13482,13483,13484,13485,13486,13487,13488,13489,13490, //13488
928 13491,13492,13493,13494,13495,13496,13497,13498,13499,13500,13501,13502,13503,13504,13505,13506, //13504
929 13507,13508,13509,13510,13511,13512,13513,13514,13515,13516,13517,13518,13519,13520,13521,13522, //13520
930 13523,13524,13525,13526,13527,13528,13529,13530,13531,13532,13533,13534,13535,13536,13537,13538, //13536
931 13539,13540,13541,13542,13543,13544,13545,13546,13547,13548,13549,13550,13551,13552,13553,13554, //13552
932 13555,13556,13557,13558,13559,13560,13561,13562,13563,13564,13565,13566,13567,13568,13569,13570, //13568
933 13571,13572,13573,13574,13575,13576,13577,13578,13579,13580,13581,13582,13583,13584,13585,13586, //13584
934 13587,13588,13589,13590,13591,13592,13593,13594,13595,13596,13597,13598,13599,13600,13601,13602, //13600
935 13603,13604,13605,13606,13607,13608,13609,13610,13611,13612,13613,13614,13615,13616,13617,13618, //13616
936 13619,13620,13621,13622,13623,13624,13625,13626,13627,13628,13629,13630,13631,13632,13633,13634, //13632
937 13635,13636,13637,13638,13639,13640,13641,13642,5006,13643,13644,13645,13646,13647,13648,13649, //13648
938 13650,13651,5007,13652,13653,13654,13655,13656,13657,13658,13659,13660,13661,13662,13663,13664, //13664
939 13665,13666,13667,13668,13669,13670,13671,13672,13673,13674,13675,13676,13677,13678,13679,13680, //13680
940 13681,13682,13683,13684,13685,13686,13687,13688,13689,13690,13691,13692,13693,13694,13695,13696, //13696
941 13697,13698,13699,13700,13701,13702,13703,13704,13705,13706,13707,13708,13709,13710,13711,13712, //13712
942 13713,13714,13715,13716,13717,13718,13719,13720,13721,13722,13723,13724,13725,13726,13727,13728, //13728
943 13729,13730,13731,13732,13733,13734,13735,13736,13737,13738,13739,13740,13741,13742,13743,13744, //13744
944 13745,13746,13747,13748,13749,13750,13751,13752,13753,13754,13755,13756,13757,13758,13759,13760, //13760
945 13761,13762,13763,13764,13765,13766,13767,13768,13769,13770,13771,13772,13773,13774,3273,13775, //13776
946 13776,13777,13778,13779,13780,13781,13782,13783,13784,13785,13786,13787,13788,13789,13790,13791, //13792
947 13792,13793,13794,13795,13796,13797,13798,13799,13800,13801,13802,13803,13804,13805,13806,13807, //13808
948 13808,13809,13810,13811,13812,13813,13814,13815,13816,13817,13818,13819,13820,13821,13822,13823, //13824
949 13824,13825,13826,13827,13828,13829,13830,13831,13832,13833,13834,13835,13836,13837,13838,13839, //13840
950 13840,13841,13842,13843,13844,13845,13846,13847,13848,13849,13850,13851,13852,13853,13854,13855, //13856
951 13856,13857,13858,13859,13860,13861,13862,13863,13864,13865,13866,13867,13868,13869,13870,13871, //13872
952 13872,13873,13874,13875,13876,13877,13878,13879,13880,13881,13882,13883,13884,13885,13886,13887, //13888
953 13888,13889,13890,13891,13892,13893,13894,13895,13896,13897,13898,13899,13900,13901,13902,13903, //13904
954 13904,13905,13906,13907,13908,13909,13910,13911,13912,13913,13914,13915,13916,13917,13918,13919, //13920
955 13920,13921,13922,13923,13924,13925,13926,13927,13928,13929,13930,13931,13932,13933,13934,13935, //13936
956 13936,13937,13938,13939,13940,13941,13942,13943,13944,13945,13946,13947,13948,13949,13950,13951, //13952
957 13952,13953,13954,13955,13956,13957,13958,13959,13960,13961,13962,13963,13964,13965,13966,13967, //13968
958 13968,13969,13970,13971,13972, //13973
959 ****************************************************************************************/
960 };
961 }
+116
-0
src/main/java/org/mozilla/universalchardet/prober/distributionanalysis/CharDistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39
40 public abstract class CharDistributionAnalysis {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final float SURE_NO = 0.01f;
45 public static final float SURE_YES = 0.99f;
46 public static final int ENOUGH_DATA_THRESHOLD = 1024;
47 public static final int MINIMUM_DATA_THRESHOLD = 4;
48
49
50 ////////////////////////////////////////////////////////////////
51 // fields
52 ////////////////////////////////////////////////////////////////
53 private int freqChars;
54 private int totalChars;
55 protected int[] charToFreqOrder; // set by subclasses
56 protected float typicalDistributionRatio; // set by subclasses
57
58
59 ////////////////////////////////////////////////////////////////
60 // methods
61 ////////////////////////////////////////////////////////////////
62 public CharDistributionAnalysis() {
63 reset();
64 }
65
66 public void handleData(final byte[] buf, int offset, int length) {
67 }
68
69 public void handleOneChar(final byte[] buf, int offset, int charLength) {
70 int order = -1;
71
72 if (charLength == 2) {
73 order = getOrder(buf, offset);
74 }
75
76 if (order >= 0) {
77 ++this.totalChars;
78 if (order < this.charToFreqOrder.length) {
79 if (512 > this.charToFreqOrder[order]) {
80 ++this.freqChars;
81 }
82 }
83 }
84 }
85
86 public float getConfidence() {
87 if (this.totalChars <= 0 || this.freqChars <= MINIMUM_DATA_THRESHOLD) {
88 return SURE_NO;
89 }
90
91 if (this.totalChars != this.freqChars) {
92 float r = this.freqChars / ((this.totalChars - this.freqChars) * this.typicalDistributionRatio);
93
94 if (r < SURE_YES) {
95 return r;
96 }
97 }
98
99 return SURE_YES;
100 }
101
102 public final void reset() {
103 this.totalChars = 0;
104 this.freqChars = 0;
105 }
106
107 public void setOption() {
108 }
109
110 public boolean gotEnoughData() {
111 return (this.totalChars > ENOUGH_DATA_THRESHOLD);
112 }
113
114 protected abstract int getOrder(final byte[] buf, int offset);
115 }
+67
-0
src/main/java/org/mozilla/universalchardet/prober/distributionanalysis/EUCJPDistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public class EUCJPDistributionAnalysis extends JISDistributionAnalysis {
40 ////////////////////////////////////////////////////////////////
41 // constants
42 ////////////////////////////////////////////////////////////////
43 public static final int HIGHBYTE_BEGIN = 0xA1;
44 public static final int HIGHBYTE_END = 0xFE;
45 public static final int LOWBYTE_BEGIN = 0xA1;
46 public static final int LOWBYTE_END = 0xFE;
47
48
49 ////////////////////////////////////////////////////////////////
50 // methods
51 ////////////////////////////////////////////////////////////////
52 public EUCJPDistributionAnalysis() {
53 super();
54 }
55
56 @Override
57 protected int getOrder(final byte[] buf, int offset) {
58 int highbyte = buf[offset] & 0xFF;
59 if (highbyte >= HIGHBYTE_BEGIN) {
60 int lowbyte = buf[offset+1] & 0xFF;
61 return (94 * (highbyte - HIGHBYTE_BEGIN) + lowbyte - LOWBYTE_BEGIN);
62 } else {
63 return -1;
64 }
65 }
66 }
+633
-0
src/main/java/org/mozilla/universalchardet/prober/distributionanalysis/EUCKRDistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public class EUCKRDistributionAnalysis extends CharDistributionAnalysis
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final float EUCKR_TYPICAL_DISTRIBUTION_RATIO = 6.0f;
45 public static final int HIGHBYTE_BEGIN = 0xB0;
46 public static final int HIGHBYTE_END = 0xFE;
47 public static final int LOWBYTE_BEGIN = 0xA1;
48 public static final int LOWBYTE_END = 0xFE;
49
50
51 ////////////////////////////////////////////////////////////////
52 // methods
53 ////////////////////////////////////////////////////////////////
54 public EUCKRDistributionAnalysis()
55 {
56 super();
57 this.typicalDistributionRatio = EUCKR_TYPICAL_DISTRIBUTION_RATIO;
58 this.charToFreqOrder = euckrCharToFreqOrder;
59 }
60
61 @Override
62 protected int getOrder(final byte[] buf, int offset)
63 {
64 int highbyte = buf[offset] & 0xFF;
65 if (highbyte >= HIGHBYTE_BEGIN) {
66 int lowbyte = buf[offset+1] & 0xFF;
67 return (94 * (highbyte - HIGHBYTE_BEGIN) + lowbyte - LOWBYTE_BEGIN);
68 } else {
69 return -1;
70 }
71 }
72
73
74 ////////////////////////////////////////////////////////////////
75 // constants continued
76 ////////////////////////////////////////////////////////////////
77 private static final int[] euckrCharToFreqOrder = new int[] {
78 13, 130, 120,1396, 481,1719,1720, 328, 609, 212,1721, 707, 400, 299,1722, 87,
79 1397,1723, 104, 536,1117,1203,1724,1267, 685,1268, 508,1725,1726,1727,1728,1398,
80 1399,1729,1730,1731, 141, 621, 326,1057, 368,1732, 267, 488, 20,1733,1269,1734,
81 945,1400,1735, 47, 904,1270,1736,1737, 773, 248,1738, 409, 313, 786, 429,1739,
82 116, 987, 813,1401, 683, 75,1204, 145,1740,1741,1742,1743, 16, 847, 667, 622,
83 708,1744,1745,1746, 966, 787, 304, 129,1747, 60, 820, 123, 676,1748,1749,1750,
84 1751, 617,1752, 626,1753,1754,1755,1756, 653,1757,1758,1759,1760,1761,1762, 856,
85 344,1763,1764,1765,1766, 89, 401, 418, 806, 905, 848,1767,1768,1769, 946,1205,
86 709,1770,1118,1771, 241,1772,1773,1774,1271,1775, 569,1776, 999,1777,1778,1779,
87 1780, 337, 751,1058, 28, 628, 254,1781, 177, 906, 270, 349, 891,1079,1782, 19,
88 1783, 379,1784, 315,1785, 629, 754,1402, 559,1786, 636, 203,1206,1787, 710, 567,
89 1788, 935, 814,1789,1790,1207, 766, 528,1791,1792,1208,1793,1794,1795,1796,1797,
90 1403,1798,1799, 533,1059,1404,1405,1156,1406, 936, 884,1080,1800, 351,1801,1802,
91 1803,1804,1805, 801,1806,1807,1808,1119,1809,1157, 714, 474,1407,1810, 298, 899,
92 885,1811,1120, 802,1158,1812, 892,1813,1814,1408, 659,1815,1816,1121,1817,1818,
93 1819,1820,1821,1822, 319,1823, 594, 545,1824, 815, 937,1209,1825,1826, 573,1409,
94 1022,1827,1210,1828,1829,1830,1831,1832,1833, 556, 722, 807,1122,1060,1834, 697,
95 1835, 900, 557, 715,1836,1410, 540,1411, 752,1159, 294, 597,1211, 976, 803, 770,
96 1412,1837,1838, 39, 794,1413, 358,1839, 371, 925,1840, 453, 661, 788, 531, 723,
97 544,1023,1081, 869, 91,1841, 392, 430, 790, 602,1414, 677,1082, 457,1415,1416,
98 1842,1843, 475, 327,1024,1417, 795, 121,1844, 733, 403,1418,1845,1846,1847, 300,
99 119, 711,1212, 627,1848,1272, 207,1849,1850, 796,1213, 382,1851, 519,1852,1083,
100 893,1853,1854,1855, 367, 809, 487, 671,1856, 663,1857,1858, 956, 471, 306, 857,
101 1859,1860,1160,1084,1861,1862,1863,1864,1865,1061,1866,1867,1868,1869,1870,1871,
102 282, 96, 574,1872, 502,1085,1873,1214,1874, 907,1875,1876, 827, 977,1419,1420,
103 1421, 268,1877,1422,1878,1879,1880, 308,1881, 2, 537,1882,1883,1215,1884,1885,
104 127, 791,1886,1273,1423,1887, 34, 336, 404, 643,1888, 571, 654, 894, 840,1889,
105 0, 886,1274, 122, 575, 260, 908, 938,1890,1275, 410, 316,1891,1892, 100,1893,
106 1894,1123, 48,1161,1124,1025,1895, 633, 901,1276,1896,1897, 115, 816,1898, 317,
107 1899, 694,1900, 909, 734,1424, 572, 866,1425, 691, 85, 524,1010, 543, 394, 841,
108 1901,1902,1903,1026,1904,1905,1906,1907,1908,1909, 30, 451, 651, 988, 310,1910,
109 1911,1426, 810,1216, 93,1912,1913,1277,1217,1914, 858, 759, 45, 58, 181, 610,
110 269,1915,1916, 131,1062, 551, 443,1000, 821,1427, 957, 895,1086,1917,1918, 375,
111 1919, 359,1920, 687,1921, 822,1922, 293,1923,1924, 40, 662, 118, 692, 29, 939,
112 887, 640, 482, 174,1925, 69,1162, 728,1428, 910,1926,1278,1218,1279, 386, 870,
113 217, 854,1163, 823,1927,1928,1929,1930, 834,1931, 78,1932, 859,1933,1063,1934,
114 1935,1936,1937, 438,1164, 208, 595,1938,1939,1940,1941,1219,1125,1942, 280, 888,
115 1429,1430,1220,1431,1943,1944,1945,1946,1947,1280, 150, 510,1432,1948,1949,1950,
116 1951,1952,1953,1954,1011,1087,1955,1433,1043,1956, 881,1957, 614, 958,1064,1065,
117 1221,1958, 638,1001, 860, 967, 896,1434, 989, 492, 553,1281,1165,1959,1282,1002,
118 1283,1222,1960,1961,1962,1963, 36, 383, 228, 753, 247, 454,1964, 876, 678,1965,
119 1966,1284, 126, 464, 490, 835, 136, 672, 529, 940,1088,1435, 473,1967,1968, 467,
120 50, 390, 227, 587, 279, 378, 598, 792, 968, 240, 151, 160, 849, 882,1126,1285,
121 639,1044, 133, 140, 288, 360, 811, 563,1027, 561, 142, 523,1969,1970,1971, 7,
122 103, 296, 439, 407, 506, 634, 990,1972,1973,1974,1975, 645,1976,1977,1978,1979,
123 1980,1981, 236,1982,1436,1983,1984,1089, 192, 828, 618, 518,1166, 333,1127,1985,
124 818,1223,1986,1987,1988,1989,1990,1991,1992,1993, 342,1128,1286, 746, 842,1994,
125 1995, 560, 223,1287, 98, 8, 189, 650, 978,1288,1996,1437,1997, 17, 345, 250,
126 423, 277, 234, 512, 226, 97, 289, 42, 167,1998, 201,1999,2000, 843, 836, 824,
127 532, 338, 783,1090, 182, 576, 436,1438,1439, 527, 500,2001, 947, 889,2002,2003,
128 2004,2005, 262, 600, 314, 447,2006, 547,2007, 693, 738,1129,2008, 71,1440, 745,
129 619, 688,2009, 829,2010,2011, 147,2012, 33, 948,2013,2014, 74, 224,2015, 61,
130 191, 918, 399, 637,2016,1028,1130, 257, 902,2017,2018,2019,2020,2021,2022,2023,
131 2024,2025,2026, 837,2027,2028,2029,2030, 179, 874, 591, 52, 724, 246,2031,2032,
132 2033,2034,1167, 969,2035,1289, 630, 605, 911,1091,1168,2036,2037,2038,1441, 912,
133 2039, 623,2040,2041, 253,1169,1290,2042,1442, 146, 620, 611, 577, 433,2043,1224,
134 719,1170, 959, 440, 437, 534, 84, 388, 480,1131, 159, 220, 198, 679,2044,1012,
135 819,1066,1443, 113,1225, 194, 318,1003,1029,2045,2046,2047,2048,1067,2049,2050,
136 2051,2052,2053, 59, 913, 112,2054, 632,2055, 455, 144, 739,1291,2056, 273, 681,
137 499,2057, 448,2058,2059, 760,2060,2061, 970, 384, 169, 245,1132,2062,2063, 414,
138 1444,2064,2065, 41, 235,2066, 157, 252, 877, 568, 919, 789, 580,2067, 725,2068,
139 2069,1292,2070,2071,1445,2072,1446,2073,2074, 55, 588, 66,1447, 271,1092,2075,
140 1226,2076, 960,1013, 372,2077,2078,2079,2080,2081,1293,2082,2083,2084,2085, 850,
141 2086,2087,2088,2089,2090, 186,2091,1068, 180,2092,2093,2094, 109,1227, 522, 606,
142 2095, 867,1448,1093, 991,1171, 926, 353,1133,2096, 581,2097,2098,2099,1294,1449,
143 1450,2100, 596,1172,1014,1228,2101,1451,1295,1173,1229,2102,2103,1296,1134,1452,
144 949,1135,2104,2105,1094,1453,1454,1455,2106,1095,2107,2108,2109,2110,2111,2112,
145 2113,2114,2115,2116,2117, 804,2118,2119,1230,1231, 805,1456, 405,1136,2120,2121,
146 2122,2123,2124, 720, 701,1297, 992,1457, 927,1004,2125,2126,2127,2128,2129,2130,
147 22, 417,2131, 303,2132, 385,2133, 971, 520, 513,2134,1174, 73,1096, 231, 274,
148 962,1458, 673,2135,1459,2136, 152,1137,2137,2138,2139,2140,1005,1138,1460,1139,
149 2141,2142,2143,2144, 11, 374, 844,2145, 154,1232, 46,1461,2146, 838, 830, 721,
150 1233, 106,2147, 90, 428, 462, 578, 566,1175, 352,2148,2149, 538,1234, 124,1298,
151 2150,1462, 761, 565,2151, 686,2152, 649,2153, 72, 173,2154, 460, 415,2155,1463,
152 2156,1235, 305,2157,2158,2159,2160,2161,2162, 579,2163,2164,2165,2166,2167, 747,
153 2168,2169,2170,2171,1464, 669,2172,2173,2174,2175,2176,1465,2177, 23, 530, 285,
154 2178, 335, 729,2179, 397,2180,2181,2182,1030,2183,2184, 698,2185,2186, 325,2187,
155 2188, 369,2189, 799,1097,1015, 348,2190,1069, 680,2191, 851,1466,2192,2193, 10,
156 2194, 613, 424,2195, 979, 108, 449, 589, 27, 172, 81,1031, 80, 774, 281, 350,
157 1032, 525, 301, 582,1176,2196, 674,1045,2197,2198,1467, 730, 762,2199,2200,2201,
158 2202,1468,2203, 993,2204,2205, 266,1070, 963,1140,2206,2207,2208, 664,1098, 972,
159 2209,2210,2211,1177,1469,1470, 871,2212,2213,2214,2215,2216,1471,2217,2218,2219,
160 2220,2221,2222,2223,2224,2225,2226,2227,1472,1236,2228,2229,2230,2231,2232,2233,
161 2234,2235,1299,2236,2237, 200,2238, 477, 373,2239,2240, 731, 825, 777,2241,2242,
162 2243, 521, 486, 548,2244,2245,2246,1473,1300, 53, 549, 137, 875, 76, 158,2247,
163 1301,1474, 469, 396,1016, 278, 712,2248, 321, 442, 503, 767, 744, 941,1237,1178,
164 1475,2249, 82, 178,1141,1179, 973,2250,1302,2251, 297,2252,2253, 570,2254,2255,
165 2256, 18, 450, 206,2257, 290, 292,1142,2258, 511, 162, 99, 346, 164, 735,2259,
166 1476,1477, 4, 554, 343, 798,1099,2260,1100,2261, 43, 171,1303, 139, 215,2262,
167 2263, 717, 775,2264,1033, 322, 216,2265, 831,2266, 149,2267,1304,2268,2269, 702,
168 1238, 135, 845, 347, 309,2270, 484,2271, 878, 655, 238,1006,1478,2272, 67,2273,
169 295,2274,2275, 461,2276, 478, 942, 412,2277,1034,2278,2279,2280, 265,2281, 541,
170 2282,2283,2284,2285,2286, 70, 852,1071,2287,2288,2289,2290, 21, 56, 509, 117,
171 432,2291,2292, 331, 980, 552,1101, 148, 284, 105, 393,1180,1239, 755,2293, 187,
172 2294,1046,1479,2295, 340,2296, 63,1047, 230,2297,2298,1305, 763,1306, 101, 800,
173 808, 494,2299,2300,2301, 903,2302, 37,1072, 14, 5,2303, 79, 675,2304, 312,
174 2305,2306,2307,2308,2309,1480, 6,1307,2310,2311,2312, 1, 470, 35, 24, 229,
175 2313, 695, 210, 86, 778, 15, 784, 592, 779, 32, 77, 855, 964,2314, 259,2315,
176 501, 380,2316,2317, 83, 981, 153, 689,1308,1481,1482,1483,2318,2319, 716,1484,
177 2320,2321,2322,2323,2324,2325,1485,2326,2327, 128, 57, 68, 261,1048, 211, 170,
178 1240, 31,2328, 51, 435, 742,2329,2330,2331, 635,2332, 264, 456,2333,2334,2335,
179 425,2336,1486, 143, 507, 263, 943,2337, 363, 920,1487, 256,1488,1102, 243, 601,
180 1489,2338,2339,2340,2341,2342,2343,2344, 861,2345,2346,2347,2348,2349,2350, 395,
181 2351,1490,1491, 62, 535, 166, 225,2352,2353, 668, 419,1241, 138, 604, 928,2354,
182 1181,2355,1492,1493,2356,2357,2358,1143,2359, 696,2360, 387, 307,1309, 682, 476,
183 2361,2362, 332, 12, 222, 156,2363, 232,2364, 641, 276, 656, 517,1494,1495,1035,
184 416, 736,1496,2365,1017, 586,2366,2367,2368,1497,2369, 242,2370,2371,2372,1498,
185 2373, 965, 713,2374,2375,2376,2377, 740, 982,1499, 944,1500,1007,2378,2379,1310,
186 1501,2380,2381,2382, 785, 329,2383,2384,1502,2385,2386,2387, 932,2388,1503,2389,
187 2390,2391,2392,1242,2393,2394,2395,2396,2397, 994, 950,2398,2399,2400,2401,1504,
188 1311,2402,2403,2404,2405,1049, 749,2406,2407, 853, 718,1144,1312,2408,1182,1505,
189 2409,2410, 255, 516, 479, 564, 550, 214,1506,1507,1313, 413, 239, 444, 339,1145,
190 1036,1508,1509,1314,1037,1510,1315,2411,1511,2412,2413,2414, 176, 703, 497, 624,
191 593, 921, 302,2415, 341, 165,1103,1512,2416,1513,2417,2418,2419, 376,2420, 700,
192 2421,2422,2423, 258, 768,1316,2424,1183,2425, 995, 608,2426,2427,2428,2429, 221,
193 2430,2431,2432,2433,2434,2435,2436,2437, 195, 323, 726, 188, 897, 983,1317, 377,
194 644,1050, 879,2438, 452,2439,2440,2441,2442,2443,2444, 914,2445,2446,2447,2448,
195 915, 489,2449,1514,1184,2450,2451, 515, 64, 427, 495,2452, 583,2453, 483, 485,
196 1038, 562, 213,1515, 748, 666,2454,2455,2456,2457, 334,2458, 780, 996,1008, 705,
197 1243,2459,2460,2461,2462,2463, 114,2464, 493,1146, 366, 163,1516, 961,1104,2465,
198 291,2466,1318,1105,2467,1517, 365,2468, 355, 951,1244,2469,1319,2470, 631,2471,
199 2472, 218,1320, 364, 320, 756,1518,1519,1321,1520,1322,2473,2474,2475,2476, 997,
200 2477,2478,2479,2480, 665,1185,2481, 916,1521,2482,2483,2484, 584, 684,2485,2486,
201 797,2487,1051,1186,2488,2489,2490,1522,2491,2492, 370,2493,1039,1187, 65,2494,
202 434, 205, 463,1188,2495, 125, 812, 391, 402, 826, 699, 286, 398, 155, 781, 771,
203 585,2496, 590, 505,1073,2497, 599, 244, 219, 917,1018, 952, 646,1523,2498,1323,
204 2499,2500, 49, 984, 354, 741,2501, 625,2502,1324,2503,1019, 190, 357, 757, 491,
205 95, 782, 868,2504,2505,2506,2507,2508,2509, 134,1524,1074, 422,1525, 898,2510,
206 161,2511,2512,2513,2514, 769,2515,1526,2516,2517, 411,1325,2518, 472,1527,2519,
207 2520,2521,2522,2523,2524, 985,2525,2526,2527,2528,2529,2530, 764,2531,1245,2532,
208 2533, 25, 204, 311,2534, 496,2535,1052,2536,2537,2538,2539,2540,2541,2542, 199,
209 704, 504, 468, 758, 657,1528, 196, 44, 839,1246, 272, 750,2543, 765, 862,2544,
210 2545,1326,2546, 132, 615, 933,2547, 732,2548,2549,2550,1189,1529,2551, 283,1247,
211 1053, 607, 929,2552,2553,2554, 930, 183, 872, 616,1040,1147,2555,1148,1020, 441,
212 249,1075,2556,2557,2558, 466, 743,2559,2560,2561, 92, 514, 426, 420, 526,2562,
213 2563,2564,2565,2566,2567,2568, 185,2569,2570,2571,2572, 776,1530, 658,2573, 362,
214 2574, 361, 922,1076, 793,2575,2576,2577,2578,2579,2580,1531, 251,2581,2582,2583,
215 2584,1532, 54, 612, 237,1327,2585,2586, 275, 408, 647, 111,2587,1533,1106, 465,
216 3, 458, 9, 38,2588, 107, 110, 890, 209, 26, 737, 498,2589,1534,2590, 431,
217 202, 88,1535, 356, 287,1107, 660,1149,2591, 381,1536, 986,1150, 445,1248,1151,
218 974,2592,2593, 846,2594, 446, 953, 184,1249,1250, 727,2595, 923, 193, 883,2596,
219 2597,2598, 102, 324, 539, 817,2599, 421,1041,2600, 832,2601, 94, 175, 197, 406,
220 2602, 459,2603,2604,2605,2606,2607, 330, 555,2608,2609,2610, 706,1108, 389,2611,
221 2612,2613,2614, 233,2615, 833, 558, 931, 954,1251,2616,2617,1537, 546,2618,2619,
222 1009,2620,2621,2622,1538, 690,1328,2623, 955,2624,1539,2625,2626, 772,2627,2628,
223 2629,2630,2631, 924, 648, 863, 603,2632,2633, 934,1540, 864, 865,2634, 642,1042,
224 670,1190,2635,2636,2637,2638, 168,2639, 652, 873, 542,1054,1541,2640,2641,2642, //512, 256
225
226 /***************************************************************************************
227 *Everything below is of no interest for detection purpose *
228 ***************************************************************************************
229
230 2643,2644,2645,2646,2647,2648,2649,2650,2651,2652,2653,2654,2655,2656,2657,2658,
231 2659,2660,2661,2662,2663,2664,2665,2666,2667,2668,2669,2670,2671,2672,2673,2674,
232 2675,2676,2677,2678,2679,2680,2681,2682,2683,2684,2685,2686,2687,2688,2689,2690,
233 2691,2692,2693,2694,2695,2696,2697,2698,2699,1542, 880,2700,2701,2702,2703,2704,
234 2705,2706,2707,2708,2709,2710,2711,2712,2713,2714,2715,2716,2717,2718,2719,2720,
235 2721,2722,2723,2724,2725,1543,2726,2727,2728,2729,2730,2731,2732,1544,2733,2734,
236 2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747,2748,2749,2750,
237 2751,2752,2753,2754,1545,2755,2756,2757,2758,2759,2760,2761,2762,2763,2764,2765,
238 2766,1546,2767,1547,2768,2769,2770,2771,2772,2773,2774,2775,2776,2777,2778,2779,
239 2780,2781,2782,2783,2784,2785,2786,1548,2787,2788,2789,1109,2790,2791,2792,2793,
240 2794,2795,2796,2797,2798,2799,2800,2801,2802,2803,2804,2805,2806,2807,2808,2809,
241 2810,2811,2812,1329,2813,2814,2815,2816,2817,2818,2819,2820,2821,2822,2823,2824,
242 2825,2826,2827,2828,2829,2830,2831,2832,2833,2834,2835,2836,2837,2838,2839,2840,
243 2841,2842,2843,2844,2845,2846,2847,2848,2849,2850,2851,2852,2853,2854,2855,2856,
244 1549,2857,2858,2859,2860,1550,2861,2862,1551,2863,2864,2865,2866,2867,2868,2869,
245 2870,2871,2872,2873,2874,1110,1330,2875,2876,2877,2878,2879,2880,2881,2882,2883,
246 2884,2885,2886,2887,2888,2889,2890,2891,2892,2893,2894,2895,2896,2897,2898,2899,
247 2900,2901,2902,2903,2904,2905,2906,2907,2908,2909,2910,2911,2912,2913,2914,2915,
248 2916,2917,2918,2919,2920,2921,2922,2923,2924,2925,2926,2927,2928,2929,2930,1331,
249 2931,2932,2933,2934,2935,2936,2937,2938,2939,2940,2941,2942,2943,1552,2944,2945,
250 2946,2947,2948,2949,2950,2951,2952,2953,2954,2955,2956,2957,2958,2959,2960,2961,
251 2962,2963,2964,1252,2965,2966,2967,2968,2969,2970,2971,2972,2973,2974,2975,2976,
252 2977,2978,2979,2980,2981,2982,2983,2984,2985,2986,2987,2988,2989,2990,2991,2992,
253 2993,2994,2995,2996,2997,2998,2999,3000,3001,3002,3003,3004,3005,3006,3007,3008,
254 3009,3010,3011,3012,1553,3013,3014,3015,3016,3017,1554,3018,1332,3019,3020,3021,
255 3022,3023,3024,3025,3026,3027,3028,3029,3030,3031,3032,3033,3034,3035,3036,3037,
256 3038,3039,3040,3041,3042,3043,3044,3045,3046,3047,3048,3049,3050,1555,3051,3052,
257 3053,1556,1557,3054,3055,3056,3057,3058,3059,3060,3061,3062,3063,3064,3065,3066,
258 3067,1558,3068,3069,3070,3071,3072,3073,3074,3075,3076,1559,3077,3078,3079,3080,
259 3081,3082,3083,1253,3084,3085,3086,3087,3088,3089,3090,3091,3092,3093,3094,3095,
260 3096,3097,3098,3099,3100,3101,3102,3103,3104,3105,3106,3107,3108,1152,3109,3110,
261 3111,3112,3113,1560,3114,3115,3116,3117,1111,3118,3119,3120,3121,3122,3123,3124,
262 3125,3126,3127,3128,3129,3130,3131,3132,3133,3134,3135,3136,3137,3138,3139,3140,
263 3141,3142,3143,3144,3145,3146,3147,3148,3149,3150,3151,3152,3153,3154,3155,3156,
264 3157,3158,3159,3160,3161,3162,3163,3164,3165,3166,3167,3168,3169,3170,3171,3172,
265 3173,3174,3175,3176,1333,3177,3178,3179,3180,3181,3182,3183,3184,3185,3186,3187,
266 3188,3189,1561,3190,3191,1334,3192,3193,3194,3195,3196,3197,3198,3199,3200,3201,
267 3202,3203,3204,3205,3206,3207,3208,3209,3210,3211,3212,3213,3214,3215,3216,3217,
268 3218,3219,3220,3221,3222,3223,3224,3225,3226,3227,3228,3229,3230,3231,3232,3233,
269 3234,1562,3235,3236,3237,3238,3239,3240,3241,3242,3243,3244,3245,3246,3247,3248,
270 3249,3250,3251,3252,3253,3254,3255,3256,3257,3258,3259,3260,3261,3262,3263,3264,
271 3265,3266,3267,3268,3269,3270,3271,3272,3273,3274,3275,3276,3277,1563,3278,3279,
272 3280,3281,3282,3283,3284,3285,3286,3287,3288,3289,3290,3291,3292,3293,3294,3295,
273 3296,3297,3298,3299,3300,3301,3302,3303,3304,3305,3306,3307,3308,3309,3310,3311,
274 3312,3313,3314,3315,3316,3317,3318,3319,3320,3321,3322,3323,3324,3325,3326,3327,
275 3328,3329,3330,3331,3332,3333,3334,3335,3336,3337,3338,3339,3340,3341,3342,3343,
276 3344,3345,3346,3347,3348,3349,3350,3351,3352,3353,3354,3355,3356,3357,3358,3359,
277 3360,3361,3362,3363,3364,1335,3365,3366,3367,3368,3369,3370,3371,3372,3373,3374,
278 3375,3376,3377,3378,3379,3380,3381,3382,3383,3384,3385,3386,3387,1336,3388,3389,
279 3390,3391,3392,3393,3394,3395,3396,3397,3398,3399,3400,3401,3402,3403,3404,3405,
280 3406,3407,3408,3409,3410,3411,3412,3413,3414,1337,3415,3416,3417,3418,3419,1338,
281 3420,3421,3422,1564,1565,3423,3424,3425,3426,3427,3428,3429,3430,3431,1254,3432,
282 3433,3434,1339,3435,3436,3437,3438,3439,1566,3440,3441,3442,3443,3444,3445,3446,
283 3447,3448,3449,3450,3451,3452,3453,3454,1255,3455,3456,3457,3458,3459,1567,1191,
284 3460,1568,1569,3461,3462,3463,1570,3464,3465,3466,3467,3468,1571,3469,3470,3471,
285 3472,3473,1572,3474,3475,3476,3477,3478,3479,3480,3481,3482,3483,3484,3485,3486,
286 1340,3487,3488,3489,3490,3491,3492,1021,3493,3494,3495,3496,3497,3498,1573,3499,
287 1341,3500,3501,3502,3503,3504,3505,3506,3507,3508,3509,3510,3511,1342,3512,3513,
288 3514,3515,3516,1574,1343,3517,3518,3519,1575,3520,1576,3521,3522,3523,3524,3525,
289 3526,3527,3528,3529,3530,3531,3532,3533,3534,3535,3536,3537,3538,3539,3540,3541,
290 3542,3543,3544,3545,3546,3547,3548,3549,3550,3551,3552,3553,3554,3555,3556,3557,
291 3558,3559,3560,3561,3562,3563,3564,3565,3566,3567,3568,3569,3570,3571,3572,3573,
292 3574,3575,3576,3577,3578,3579,3580,1577,3581,3582,1578,3583,3584,3585,3586,3587,
293 3588,3589,3590,3591,3592,3593,3594,3595,3596,3597,3598,3599,3600,3601,3602,3603,
294 3604,1579,3605,3606,3607,3608,3609,3610,3611,3612,3613,3614,3615,3616,3617,3618,
295 3619,3620,3621,3622,3623,3624,3625,3626,3627,3628,3629,1580,3630,3631,1581,3632,
296 3633,3634,3635,3636,3637,3638,3639,3640,3641,3642,3643,3644,3645,3646,3647,3648,
297 3649,3650,3651,3652,3653,3654,3655,3656,1582,3657,3658,3659,3660,3661,3662,3663,
298 3664,3665,3666,3667,3668,3669,3670,3671,3672,3673,3674,3675,3676,3677,3678,3679,
299 3680,3681,3682,3683,3684,3685,3686,3687,3688,3689,3690,3691,3692,3693,3694,3695,
300 3696,3697,3698,3699,3700,1192,3701,3702,3703,3704,1256,3705,3706,3707,3708,1583,
301 1257,3709,3710,3711,3712,3713,3714,3715,3716,1584,3717,3718,3719,3720,3721,3722,
302 3723,3724,3725,3726,3727,3728,3729,3730,3731,3732,3733,3734,3735,3736,3737,3738,
303 3739,3740,3741,3742,3743,3744,3745,1344,3746,3747,3748,3749,3750,3751,3752,3753,
304 3754,3755,3756,1585,3757,3758,3759,3760,3761,3762,3763,3764,3765,3766,1586,3767,
305 3768,3769,3770,3771,3772,3773,3774,3775,3776,3777,3778,1345,3779,3780,3781,3782,
306 3783,3784,3785,3786,3787,3788,3789,3790,3791,3792,3793,3794,3795,1346,1587,3796,
307 3797,1588,3798,3799,3800,3801,3802,3803,3804,3805,3806,1347,3807,3808,3809,3810,
308 3811,1589,3812,3813,3814,3815,3816,3817,3818,3819,3820,3821,1590,3822,3823,1591,
309 1348,3824,3825,3826,3827,3828,3829,3830,1592,3831,3832,1593,3833,3834,3835,3836,
310 3837,3838,3839,3840,3841,3842,3843,3844,1349,3845,3846,3847,3848,3849,3850,3851,
311 3852,3853,3854,3855,3856,3857,3858,1594,3859,3860,3861,3862,3863,3864,3865,3866,
312 3867,3868,3869,1595,3870,3871,3872,3873,1596,3874,3875,3876,3877,3878,3879,3880,
313 3881,3882,3883,3884,3885,3886,1597,3887,3888,3889,3890,3891,3892,3893,3894,3895,
314 1598,3896,3897,3898,1599,1600,3899,1350,3900,1351,3901,3902,1352,3903,3904,3905,
315 3906,3907,3908,3909,3910,3911,3912,3913,3914,3915,3916,3917,3918,3919,3920,3921,
316 3922,3923,3924,1258,3925,3926,3927,3928,3929,3930,3931,1193,3932,1601,3933,3934,
317 3935,3936,3937,3938,3939,3940,3941,3942,3943,1602,3944,3945,3946,3947,3948,1603,
318 3949,3950,3951,3952,3953,3954,3955,3956,3957,3958,3959,3960,3961,3962,3963,3964,
319 3965,1604,3966,3967,3968,3969,3970,3971,3972,3973,3974,3975,3976,3977,1353,3978,
320 3979,3980,3981,3982,3983,3984,3985,3986,3987,3988,3989,3990,3991,1354,3992,3993,
321 3994,3995,3996,3997,3998,3999,4000,4001,4002,4003,4004,4005,4006,4007,4008,4009,
322 4010,4011,4012,4013,4014,4015,4016,4017,4018,4019,4020,4021,4022,4023,1355,4024,
323 4025,4026,4027,4028,4029,4030,4031,4032,4033,4034,4035,4036,4037,4038,4039,4040,
324 1605,4041,4042,4043,4044,4045,4046,4047,4048,4049,4050,4051,4052,4053,4054,4055,
325 4056,4057,4058,4059,4060,1606,4061,4062,4063,4064,1607,4065,4066,4067,4068,4069,
326 4070,4071,4072,4073,4074,4075,4076,1194,4077,4078,1608,4079,4080,4081,4082,4083,
327 4084,4085,4086,4087,1609,4088,4089,4090,4091,4092,4093,4094,4095,4096,4097,4098,
328 4099,4100,4101,4102,4103,4104,4105,4106,4107,4108,1259,4109,4110,4111,4112,4113,
329 4114,4115,4116,4117,4118,4119,4120,4121,4122,4123,4124,1195,4125,4126,4127,1610,
330 4128,4129,4130,4131,4132,4133,4134,4135,4136,4137,1356,4138,4139,4140,4141,4142,
331 4143,4144,1611,4145,4146,4147,4148,4149,4150,4151,4152,4153,4154,4155,4156,4157,
332 4158,4159,4160,4161,4162,4163,4164,4165,4166,4167,4168,4169,4170,4171,4172,4173,
333 4174,4175,4176,4177,4178,4179,4180,4181,4182,4183,4184,4185,4186,4187,4188,4189,
334 4190,4191,4192,4193,4194,4195,4196,4197,4198,4199,4200,4201,4202,4203,4204,4205,
335 4206,4207,4208,4209,4210,4211,4212,4213,4214,4215,4216,4217,4218,4219,1612,4220,
336 4221,4222,4223,4224,4225,4226,4227,1357,4228,1613,4229,4230,4231,4232,4233,4234,
337 4235,4236,4237,4238,4239,4240,4241,4242,4243,1614,4244,4245,4246,4247,4248,4249,
338 4250,4251,4252,4253,4254,4255,4256,4257,4258,4259,4260,4261,4262,4263,4264,4265,
339 4266,4267,4268,4269,4270,1196,1358,4271,4272,4273,4274,4275,4276,4277,4278,4279,
340 4280,4281,4282,4283,4284,4285,4286,4287,1615,4288,4289,4290,4291,4292,4293,4294,
341 4295,4296,4297,4298,4299,4300,4301,4302,4303,4304,4305,4306,4307,4308,4309,4310,
342 4311,4312,4313,4314,4315,4316,4317,4318,4319,4320,4321,4322,4323,4324,4325,4326,
343 4327,4328,4329,4330,4331,4332,4333,4334,1616,4335,4336,4337,4338,4339,4340,4341,
344 4342,4343,4344,4345,4346,4347,4348,4349,4350,4351,4352,4353,4354,4355,4356,4357,
345 4358,4359,4360,1617,4361,4362,4363,4364,4365,1618,4366,4367,4368,4369,4370,4371,
346 4372,4373,4374,4375,4376,4377,4378,4379,4380,4381,4382,4383,4384,4385,4386,4387,
347 4388,4389,4390,4391,4392,4393,4394,4395,4396,4397,4398,4399,4400,4401,4402,4403,
348 4404,4405,4406,4407,4408,4409,4410,4411,4412,4413,4414,4415,4416,1619,4417,4418,
349 4419,4420,4421,4422,4423,4424,4425,1112,4426,4427,4428,4429,4430,1620,4431,4432,
350 4433,4434,4435,4436,4437,4438,4439,4440,4441,4442,1260,1261,4443,4444,4445,4446,
351 4447,4448,4449,4450,4451,4452,4453,4454,4455,1359,4456,4457,4458,4459,4460,4461,
352 4462,4463,4464,4465,1621,4466,4467,4468,4469,4470,4471,4472,4473,4474,4475,4476,
353 4477,4478,4479,4480,4481,4482,4483,4484,4485,4486,4487,4488,4489,1055,4490,4491,
354 4492,4493,4494,4495,4496,4497,4498,4499,4500,4501,4502,4503,4504,4505,4506,4507,
355 4508,4509,4510,4511,4512,4513,4514,4515,4516,4517,4518,1622,4519,4520,4521,1623,
356 4522,4523,4524,4525,4526,4527,4528,4529,4530,4531,4532,4533,4534,4535,1360,4536,
357 4537,4538,4539,4540,4541,4542,4543, 975,4544,4545,4546,4547,4548,4549,4550,4551,
358 4552,4553,4554,4555,4556,4557,4558,4559,4560,4561,4562,4563,4564,4565,4566,4567,
359 4568,4569,4570,4571,1624,4572,4573,4574,4575,4576,1625,4577,4578,4579,4580,4581,
360 4582,4583,4584,1626,4585,4586,4587,4588,4589,4590,4591,4592,4593,4594,4595,1627,
361 4596,4597,4598,4599,4600,4601,4602,4603,4604,4605,4606,4607,4608,4609,4610,4611,
362 4612,4613,4614,4615,1628,4616,4617,4618,4619,4620,4621,4622,4623,4624,4625,4626,
363 4627,4628,4629,4630,4631,4632,4633,4634,4635,4636,4637,4638,4639,4640,4641,4642,
364 4643,4644,4645,4646,4647,4648,4649,1361,4650,4651,4652,4653,4654,4655,4656,4657,
365 4658,4659,4660,4661,1362,4662,4663,4664,4665,4666,4667,4668,4669,4670,4671,4672,
366 4673,4674,4675,4676,4677,4678,4679,4680,4681,4682,1629,4683,4684,4685,4686,4687,
367 1630,4688,4689,4690,4691,1153,4692,4693,4694,1113,4695,4696,4697,4698,4699,4700,
368 4701,4702,4703,4704,4705,4706,4707,4708,4709,4710,4711,1197,4712,4713,4714,4715,
369 4716,4717,4718,4719,4720,4721,4722,4723,4724,4725,4726,4727,4728,4729,4730,4731,
370 4732,4733,4734,4735,1631,4736,1632,4737,4738,4739,4740,4741,4742,4743,4744,1633,
371 4745,4746,4747,4748,4749,1262,4750,4751,4752,4753,4754,1363,4755,4756,4757,4758,
372 4759,4760,4761,4762,4763,4764,4765,4766,4767,4768,1634,4769,4770,4771,4772,4773,
373 4774,4775,4776,4777,4778,1635,4779,4780,4781,4782,4783,4784,4785,4786,4787,4788,
374 4789,1636,4790,4791,4792,4793,4794,4795,4796,4797,4798,4799,4800,4801,4802,4803,
375 4804,4805,4806,1637,4807,4808,4809,1638,4810,4811,4812,4813,4814,4815,4816,4817,
376 4818,1639,4819,4820,4821,4822,4823,4824,4825,4826,4827,4828,4829,4830,4831,4832,
377 4833,1077,4834,4835,4836,4837,4838,4839,4840,4841,4842,4843,4844,4845,4846,4847,
378 4848,4849,4850,4851,4852,4853,4854,4855,4856,4857,4858,4859,4860,4861,4862,4863,
379 4864,4865,4866,4867,4868,4869,4870,4871,4872,4873,4874,4875,4876,4877,4878,4879,
380 4880,4881,4882,4883,1640,4884,4885,1641,4886,4887,4888,4889,4890,4891,4892,4893,
381 4894,4895,4896,4897,4898,4899,4900,4901,4902,4903,4904,4905,4906,4907,4908,4909,
382 4910,4911,1642,4912,4913,4914,1364,4915,4916,4917,4918,4919,4920,4921,4922,4923,
383 4924,4925,4926,4927,4928,4929,4930,4931,1643,4932,4933,4934,4935,4936,4937,4938,
384 4939,4940,4941,4942,4943,4944,4945,4946,4947,4948,4949,4950,4951,4952,4953,4954,
385 4955,4956,4957,4958,4959,4960,4961,4962,4963,4964,4965,4966,4967,4968,4969,4970,
386 4971,4972,4973,4974,4975,4976,4977,4978,4979,4980,1644,4981,4982,4983,4984,1645,
387 4985,4986,1646,4987,4988,4989,4990,4991,4992,4993,4994,4995,4996,4997,4998,4999,
388 5000,5001,5002,5003,5004,5005,1647,5006,1648,5007,5008,5009,5010,5011,5012,1078,
389 5013,5014,5015,5016,5017,5018,5019,5020,5021,5022,5023,5024,5025,5026,5027,5028,
390 1365,5029,5030,5031,5032,5033,5034,5035,5036,5037,5038,5039,1649,5040,5041,5042,
391 5043,5044,5045,1366,5046,5047,5048,5049,5050,5051,5052,5053,5054,5055,1650,5056,
392 5057,5058,5059,5060,5061,5062,5063,5064,5065,5066,5067,5068,5069,5070,5071,5072,
393 5073,5074,5075,5076,5077,1651,5078,5079,5080,5081,5082,5083,5084,5085,5086,5087,
394 5088,5089,5090,5091,5092,5093,5094,5095,5096,5097,5098,5099,5100,5101,5102,5103,
395 5104,5105,5106,5107,5108,5109,5110,1652,5111,5112,5113,5114,5115,5116,5117,5118,
396 1367,5119,5120,5121,5122,5123,5124,5125,5126,5127,5128,5129,1653,5130,5131,5132,
397 5133,5134,5135,5136,5137,5138,5139,5140,5141,5142,5143,5144,5145,5146,5147,5148,
398 5149,1368,5150,1654,5151,1369,5152,5153,5154,5155,5156,5157,5158,5159,5160,5161,
399 5162,5163,5164,5165,5166,5167,5168,5169,5170,5171,5172,5173,5174,5175,5176,5177,
400 5178,1370,5179,5180,5181,5182,5183,5184,5185,5186,5187,5188,5189,5190,5191,5192,
401 5193,5194,5195,5196,5197,5198,1655,5199,5200,5201,5202,1656,5203,5204,5205,5206,
402 1371,5207,1372,5208,5209,5210,5211,1373,5212,5213,1374,5214,5215,5216,5217,5218,
403 5219,5220,5221,5222,5223,5224,5225,5226,5227,5228,5229,5230,5231,5232,5233,5234,
404 5235,5236,5237,5238,5239,5240,5241,5242,5243,5244,5245,5246,5247,1657,5248,5249,
405 5250,5251,1658,1263,5252,5253,5254,5255,5256,1375,5257,5258,5259,5260,5261,5262,
406 5263,5264,5265,5266,5267,5268,5269,5270,5271,5272,5273,5274,5275,5276,5277,5278,
407 5279,5280,5281,5282,5283,1659,5284,5285,5286,5287,5288,5289,5290,5291,5292,5293,
408 5294,5295,5296,5297,5298,5299,5300,1660,5301,5302,5303,5304,5305,5306,5307,5308,
409 5309,5310,5311,5312,5313,5314,5315,5316,5317,5318,5319,5320,5321,1376,5322,5323,
410 5324,5325,5326,5327,5328,5329,5330,5331,5332,5333,1198,5334,5335,5336,5337,5338,
411 5339,5340,5341,5342,5343,1661,5344,5345,5346,5347,5348,5349,5350,5351,5352,5353,
412 5354,5355,5356,5357,5358,5359,5360,5361,5362,5363,5364,5365,5366,5367,5368,5369,
413 5370,5371,5372,5373,5374,5375,5376,5377,5378,5379,5380,5381,5382,5383,5384,5385,
414 5386,5387,5388,5389,5390,5391,5392,5393,5394,5395,5396,5397,5398,1264,5399,5400,
415 5401,5402,5403,5404,5405,5406,5407,5408,5409,5410,5411,5412,1662,5413,5414,5415,
416 5416,1663,5417,5418,5419,5420,5421,5422,5423,5424,5425,5426,5427,5428,5429,5430,
417 5431,5432,5433,5434,5435,5436,5437,5438,1664,5439,5440,5441,5442,5443,5444,5445,
418 5446,5447,5448,5449,5450,5451,5452,5453,5454,5455,5456,5457,5458,5459,5460,5461,
419 5462,5463,5464,5465,5466,5467,5468,5469,5470,5471,5472,5473,5474,5475,5476,5477,
420 5478,1154,5479,5480,5481,5482,5483,5484,5485,1665,5486,5487,5488,5489,5490,5491,
421 5492,5493,5494,5495,5496,5497,5498,5499,5500,5501,5502,5503,5504,5505,5506,5507,
422 5508,5509,5510,5511,5512,5513,5514,5515,5516,5517,5518,5519,5520,5521,5522,5523,
423 5524,5525,5526,5527,5528,5529,5530,5531,5532,5533,5534,5535,5536,5537,5538,5539,
424 5540,5541,5542,5543,5544,5545,5546,5547,5548,1377,5549,5550,5551,5552,5553,5554,
425 5555,5556,5557,5558,5559,5560,5561,5562,5563,5564,5565,5566,5567,5568,5569,5570,
426 1114,5571,5572,5573,5574,5575,5576,5577,5578,5579,5580,5581,5582,5583,5584,5585,
427 5586,5587,5588,5589,5590,5591,5592,1378,5593,5594,5595,5596,5597,5598,5599,5600,
428 5601,5602,5603,5604,5605,5606,5607,5608,5609,5610,5611,5612,5613,5614,1379,5615,
429 5616,5617,5618,5619,5620,5621,5622,5623,5624,5625,5626,5627,5628,5629,5630,5631,
430 5632,5633,5634,1380,5635,5636,5637,5638,5639,5640,5641,5642,5643,5644,5645,5646,
431 5647,5648,5649,1381,1056,5650,5651,5652,5653,5654,5655,5656,5657,5658,5659,5660,
432 1666,5661,5662,5663,5664,5665,5666,5667,5668,1667,5669,1668,5670,5671,5672,5673,
433 5674,5675,5676,5677,5678,1155,5679,5680,5681,5682,5683,5684,5685,5686,5687,5688,
434 5689,5690,5691,5692,5693,5694,5695,5696,5697,5698,1669,5699,5700,5701,5702,5703,
435 5704,5705,1670,5706,5707,5708,5709,5710,1671,5711,5712,5713,5714,1382,5715,5716,
436 5717,5718,5719,5720,5721,5722,5723,5724,5725,1672,5726,5727,1673,1674,5728,5729,
437 5730,5731,5732,5733,5734,5735,5736,1675,5737,5738,5739,5740,5741,5742,5743,5744,
438 1676,5745,5746,5747,5748,5749,5750,5751,1383,5752,5753,5754,5755,5756,5757,5758,
439 5759,5760,5761,5762,5763,5764,5765,5766,5767,5768,1677,5769,5770,5771,5772,5773,
440 1678,5774,5775,5776, 998,5777,5778,5779,5780,5781,5782,5783,5784,5785,1384,5786,
441 5787,5788,5789,5790,5791,5792,5793,5794,5795,5796,5797,5798,5799,5800,1679,5801,
442 5802,5803,1115,1116,5804,5805,5806,5807,5808,5809,5810,5811,5812,5813,5814,5815,
443 5816,5817,5818,5819,5820,5821,5822,5823,5824,5825,5826,5827,5828,5829,5830,5831,
444 5832,5833,5834,5835,5836,5837,5838,5839,5840,5841,5842,5843,5844,5845,5846,5847,
445 5848,5849,5850,5851,5852,5853,5854,5855,1680,5856,5857,5858,5859,5860,5861,5862,
446 5863,5864,1681,5865,5866,5867,1682,5868,5869,5870,5871,5872,5873,5874,5875,5876,
447 5877,5878,5879,1683,5880,1684,5881,5882,5883,5884,1685,5885,5886,5887,5888,5889,
448 5890,5891,5892,5893,5894,5895,5896,5897,5898,5899,5900,5901,5902,5903,5904,5905,
449 5906,5907,1686,5908,5909,5910,5911,5912,5913,5914,5915,5916,5917,5918,5919,5920,
450 5921,5922,5923,5924,5925,5926,5927,5928,5929,5930,5931,5932,5933,5934,5935,1687,
451 5936,5937,5938,5939,5940,5941,5942,5943,5944,5945,5946,5947,5948,5949,5950,5951,
452 5952,1688,1689,5953,1199,5954,5955,5956,5957,5958,5959,5960,5961,1690,5962,5963,
453 5964,5965,5966,5967,5968,5969,5970,5971,5972,5973,5974,5975,5976,5977,5978,5979,
454 5980,5981,1385,5982,1386,5983,5984,5985,5986,5987,5988,5989,5990,5991,5992,5993,
455 5994,5995,5996,5997,5998,5999,6000,6001,6002,6003,6004,6005,6006,6007,6008,6009,
456 6010,6011,6012,6013,6014,6015,6016,6017,6018,6019,6020,6021,6022,6023,6024,6025,
457 6026,6027,1265,6028,6029,1691,6030,6031,6032,6033,6034,6035,6036,6037,6038,6039,
458 6040,6041,6042,6043,6044,6045,6046,6047,6048,6049,6050,6051,6052,6053,6054,6055,
459 6056,6057,6058,6059,6060,6061,6062,6063,6064,6065,6066,6067,6068,6069,6070,6071,
460 6072,6073,6074,6075,6076,6077,6078,6079,6080,6081,6082,6083,6084,1692,6085,6086,
461 6087,6088,6089,6090,6091,6092,6093,6094,6095,6096,6097,6098,6099,6100,6101,6102,
462 6103,6104,6105,6106,6107,6108,6109,6110,6111,6112,6113,6114,6115,6116,6117,6118,
463 6119,6120,6121,6122,6123,6124,6125,6126,6127,6128,6129,6130,6131,1693,6132,6133,
464 6134,6135,6136,1694,6137,6138,6139,6140,6141,1695,6142,6143,6144,6145,6146,6147,
465 6148,6149,6150,6151,6152,6153,6154,6155,6156,6157,6158,6159,6160,6161,6162,6163,
466 6164,6165,6166,6167,6168,6169,6170,6171,6172,6173,6174,6175,6176,6177,6178,6179,
467 6180,6181,6182,6183,6184,6185,1696,6186,6187,6188,6189,6190,6191,6192,6193,6194,
468 6195,6196,6197,6198,6199,6200,6201,6202,6203,6204,6205,6206,6207,6208,6209,6210,
469 6211,6212,6213,6214,6215,6216,6217,6218,6219,1697,6220,6221,6222,6223,6224,6225,
470 6226,6227,6228,6229,6230,6231,6232,6233,6234,6235,6236,6237,6238,6239,6240,6241,
471 6242,6243,6244,6245,6246,6247,6248,6249,6250,6251,6252,6253,1698,6254,6255,6256,
472 6257,6258,6259,6260,6261,6262,6263,1200,6264,6265,6266,6267,6268,6269,6270,6271, //1024
473 6272,6273,6274,6275,6276,6277,6278,6279,6280,6281,6282,6283,6284,6285,6286,6287,
474 6288,6289,6290,6291,6292,6293,6294,6295,6296,6297,6298,6299,6300,6301,6302,1699,
475 6303,6304,1700,6305,6306,6307,6308,6309,6310,6311,6312,6313,6314,6315,6316,6317,
476 6318,6319,6320,6321,6322,6323,6324,6325,6326,6327,6328,6329,6330,6331,6332,6333,
477 6334,6335,6336,6337,6338,6339,1701,6340,6341,6342,6343,6344,1387,6345,6346,6347,
478 6348,6349,6350,6351,6352,6353,6354,6355,6356,6357,6358,6359,6360,6361,6362,6363,
479 6364,6365,6366,6367,6368,6369,6370,6371,6372,6373,6374,6375,6376,6377,6378,6379,
480 6380,6381,6382,6383,6384,6385,6386,6387,6388,6389,6390,6391,6392,6393,6394,6395,
481 6396,6397,6398,6399,6400,6401,6402,6403,6404,6405,6406,6407,6408,6409,6410,6411,
482 6412,6413,1702,6414,6415,6416,6417,6418,6419,6420,6421,6422,1703,6423,6424,6425,
483 6426,6427,6428,6429,6430,6431,6432,6433,6434,6435,6436,6437,6438,1704,6439,6440,
484 6441,6442,6443,6444,6445,6446,6447,6448,6449,6450,6451,6452,6453,6454,6455,6456,
485 6457,6458,6459,6460,6461,6462,6463,6464,6465,6466,6467,6468,6469,6470,6471,6472,
486 6473,6474,6475,6476,6477,6478,6479,6480,6481,6482,6483,6484,6485,6486,6487,6488,
487 6489,6490,6491,6492,6493,6494,6495,6496,6497,6498,6499,6500,6501,6502,6503,1266,
488 6504,6505,6506,6507,6508,6509,6510,6511,6512,6513,6514,6515,6516,6517,6518,6519,
489 6520,6521,6522,6523,6524,6525,6526,6527,6528,6529,6530,6531,6532,6533,6534,6535,
490 6536,6537,6538,6539,6540,6541,6542,6543,6544,6545,6546,6547,6548,6549,6550,6551,
491 1705,1706,6552,6553,6554,6555,6556,6557,6558,6559,6560,6561,6562,6563,6564,6565,
492 6566,6567,6568,6569,6570,6571,6572,6573,6574,6575,6576,6577,6578,6579,6580,6581,
493 6582,6583,6584,6585,6586,6587,6588,6589,6590,6591,6592,6593,6594,6595,6596,6597,
494 6598,6599,6600,6601,6602,6603,6604,6605,6606,6607,6608,6609,6610,6611,6612,6613,
495 6614,6615,6616,6617,6618,6619,6620,6621,6622,6623,6624,6625,6626,6627,6628,6629,
496 6630,6631,6632,6633,6634,6635,6636,6637,1388,6638,6639,6640,6641,6642,6643,6644,
497 1707,6645,6646,6647,6648,6649,6650,6651,6652,6653,6654,6655,6656,6657,6658,6659,
498 6660,6661,6662,6663,1708,6664,6665,6666,6667,6668,6669,6670,6671,6672,6673,6674,
499 1201,6675,6676,6677,6678,6679,6680,6681,6682,6683,6684,6685,6686,6687,6688,6689,
500 6690,6691,6692,6693,6694,6695,6696,6697,6698,6699,6700,6701,6702,6703,6704,6705,
501 6706,6707,6708,6709,6710,6711,6712,6713,6714,6715,6716,6717,6718,6719,6720,6721,
502 6722,6723,6724,6725,1389,6726,6727,6728,6729,6730,6731,6732,6733,6734,6735,6736,
503 1390,1709,6737,6738,6739,6740,6741,6742,1710,6743,6744,6745,6746,1391,6747,6748,
504 6749,6750,6751,6752,6753,6754,6755,6756,6757,1392,6758,6759,6760,6761,6762,6763,
505 6764,6765,6766,6767,6768,6769,6770,6771,6772,6773,6774,6775,6776,6777,6778,6779,
506 6780,1202,6781,6782,6783,6784,6785,6786,6787,6788,6789,6790,6791,6792,6793,6794,
507 6795,6796,6797,6798,6799,6800,6801,6802,6803,6804,6805,6806,6807,6808,6809,1711,
508 6810,6811,6812,6813,6814,6815,6816,6817,6818,6819,6820,6821,6822,6823,6824,6825,
509 6826,6827,6828,6829,6830,6831,6832,6833,6834,6835,6836,1393,6837,6838,6839,6840,
510 6841,6842,6843,6844,6845,6846,6847,6848,6849,6850,6851,6852,6853,6854,6855,6856,
511 6857,6858,6859,6860,6861,6862,6863,6864,6865,6866,6867,6868,6869,6870,6871,6872,
512 6873,6874,6875,6876,6877,6878,6879,6880,6881,6882,6883,6884,6885,6886,6887,6888,
513 6889,6890,6891,6892,6893,6894,6895,6896,6897,6898,6899,6900,6901,6902,1712,6903,
514 6904,6905,6906,6907,6908,6909,6910,1713,6911,6912,6913,6914,6915,6916,6917,6918,
515 6919,6920,6921,6922,6923,6924,6925,6926,6927,6928,6929,6930,6931,6932,6933,6934,
516 6935,6936,6937,6938,6939,6940,6941,6942,6943,6944,6945,6946,6947,6948,6949,6950,
517 6951,6952,6953,6954,6955,6956,6957,6958,6959,6960,6961,6962,6963,6964,6965,6966,
518 6967,6968,6969,6970,6971,6972,6973,6974,1714,6975,6976,6977,6978,6979,6980,6981,
519 6982,6983,6984,6985,6986,6987,6988,1394,6989,6990,6991,6992,6993,6994,6995,6996,
520 6997,6998,6999,7000,1715,7001,7002,7003,7004,7005,7006,7007,7008,7009,7010,7011,
521 7012,7013,7014,7015,7016,7017,7018,7019,7020,7021,7022,7023,7024,7025,7026,7027,
522 7028,1716,7029,7030,7031,7032,7033,7034,7035,7036,7037,7038,7039,7040,7041,7042,
523 7043,7044,7045,7046,7047,7048,7049,7050,7051,7052,7053,7054,7055,7056,7057,7058,
524 7059,7060,7061,7062,7063,7064,7065,7066,7067,7068,7069,7070,7071,7072,7073,7074,
525 7075,7076,7077,7078,7079,7080,7081,7082,7083,7084,7085,7086,7087,7088,7089,7090,
526 7091,7092,7093,7094,7095,7096,7097,7098,7099,7100,7101,7102,7103,7104,7105,7106,
527 7107,7108,7109,7110,7111,7112,7113,7114,7115,7116,7117,7118,7119,7120,7121,7122,
528 7123,7124,7125,7126,7127,7128,7129,7130,7131,7132,7133,7134,7135,7136,7137,7138,
529 7139,7140,7141,7142,7143,7144,7145,7146,7147,7148,7149,7150,7151,7152,7153,7154,
530 7155,7156,7157,7158,7159,7160,7161,7162,7163,7164,7165,7166,7167,7168,7169,7170,
531 7171,7172,7173,7174,7175,7176,7177,7178,7179,7180,7181,7182,7183,7184,7185,7186,
532 7187,7188,7189,7190,7191,7192,7193,7194,7195,7196,7197,7198,7199,7200,7201,7202,
533 7203,7204,7205,7206,7207,1395,7208,7209,7210,7211,7212,7213,1717,7214,7215,7216,
534 7217,7218,7219,7220,7221,7222,7223,7224,7225,7226,7227,7228,7229,7230,7231,7232,
535 7233,7234,7235,7236,7237,7238,7239,7240,7241,7242,7243,7244,7245,7246,7247,7248,
536 7249,7250,7251,7252,7253,7254,7255,7256,7257,7258,7259,7260,7261,7262,7263,7264,
537 7265,7266,7267,7268,7269,7270,7271,7272,7273,7274,7275,7276,7277,7278,7279,7280,
538 7281,7282,7283,7284,7285,7286,7287,7288,7289,7290,7291,7292,7293,7294,7295,7296,
539 7297,7298,7299,7300,7301,7302,7303,7304,7305,7306,7307,7308,7309,7310,7311,7312,
540 7313,1718,7314,7315,7316,7317,7318,7319,7320,7321,7322,7323,7324,7325,7326,7327,
541 7328,7329,7330,7331,7332,7333,7334,7335,7336,7337,7338,7339,7340,7341,7342,7343,
542 7344,7345,7346,7347,7348,7349,7350,7351,7352,7353,7354,7355,7356,7357,7358,7359,
543 7360,7361,7362,7363,7364,7365,7366,7367,7368,7369,7370,7371,7372,7373,7374,7375,
544 7376,7377,7378,7379,7380,7381,7382,7383,7384,7385,7386,7387,7388,7389,7390,7391,
545 7392,7393,7394,7395,7396,7397,7398,7399,7400,7401,7402,7403,7404,7405,7406,7407,
546 7408,7409,7410,7411,7412,7413,7414,7415,7416,7417,7418,7419,7420,7421,7422,7423,
547 7424,7425,7426,7427,7428,7429,7430,7431,7432,7433,7434,7435,7436,7437,7438,7439,
548 7440,7441,7442,7443,7444,7445,7446,7447,7448,7449,7450,7451,7452,7453,7454,7455,
549 7456,7457,7458,7459,7460,7461,7462,7463,7464,7465,7466,7467,7468,7469,7470,7471,
550 7472,7473,7474,7475,7476,7477,7478,7479,7480,7481,7482,7483,7484,7485,7486,7487,
551 7488,7489,7490,7491,7492,7493,7494,7495,7496,7497,7498,7499,7500,7501,7502,7503,
552 7504,7505,7506,7507,7508,7509,7510,7511,7512,7513,7514,7515,7516,7517,7518,7519,
553 7520,7521,7522,7523,7524,7525,7526,7527,7528,7529,7530,7531,7532,7533,7534,7535,
554 7536,7537,7538,7539,7540,7541,7542,7543,7544,7545,7546,7547,7548,7549,7550,7551,
555 7552,7553,7554,7555,7556,7557,7558,7559,7560,7561,7562,7563,7564,7565,7566,7567,
556 7568,7569,7570,7571,7572,7573,7574,7575,7576,7577,7578,7579,7580,7581,7582,7583,
557 7584,7585,7586,7587,7588,7589,7590,7591,7592,7593,7594,7595,7596,7597,7598,7599,
558 7600,7601,7602,7603,7604,7605,7606,7607,7608,7609,7610,7611,7612,7613,7614,7615,
559 7616,7617,7618,7619,7620,7621,7622,7623,7624,7625,7626,7627,7628,7629,7630,7631,
560 7632,7633,7634,7635,7636,7637,7638,7639,7640,7641,7642,7643,7644,7645,7646,7647,
561 7648,7649,7650,7651,7652,7653,7654,7655,7656,7657,7658,7659,7660,7661,7662,7663,
562 7664,7665,7666,7667,7668,7669,7670,7671,7672,7673,7674,7675,7676,7677,7678,7679,
563 7680,7681,7682,7683,7684,7685,7686,7687,7688,7689,7690,7691,7692,7693,7694,7695,
564 7696,7697,7698,7699,7700,7701,7702,7703,7704,7705,7706,7707,7708,7709,7710,7711,
565 7712,7713,7714,7715,7716,7717,7718,7719,7720,7721,7722,7723,7724,7725,7726,7727,
566 7728,7729,7730,7731,7732,7733,7734,7735,7736,7737,7738,7739,7740,7741,7742,7743,
567 7744,7745,7746,7747,7748,7749,7750,7751,7752,7753,7754,7755,7756,7757,7758,7759,
568 7760,7761,7762,7763,7764,7765,7766,7767,7768,7769,7770,7771,7772,7773,7774,7775,
569 7776,7777,7778,7779,7780,7781,7782,7783,7784,7785,7786,7787,7788,7789,7790,7791,
570 7792,7793,7794,7795,7796,7797,7798,7799,7800,7801,7802,7803,7804,7805,7806,7807,
571 7808,7809,7810,7811,7812,7813,7814,7815,7816,7817,7818,7819,7820,7821,7822,7823,
572 7824,7825,7826,7827,7828,7829,7830,7831,7832,7833,7834,7835,7836,7837,7838,7839,
573 7840,7841,7842,7843,7844,7845,7846,7847,7848,7849,7850,7851,7852,7853,7854,7855,
574 7856,7857,7858,7859,7860,7861,7862,7863,7864,7865,7866,7867,7868,7869,7870,7871,
575 7872,7873,7874,7875,7876,7877,7878,7879,7880,7881,7882,7883,7884,7885,7886,7887,
576 7888,7889,7890,7891,7892,7893,7894,7895,7896,7897,7898,7899,7900,7901,7902,7903,
577 7904,7905,7906,7907,7908,7909,7910,7911,7912,7913,7914,7915,7916,7917,7918,7919,
578 7920,7921,7922,7923,7924,7925,7926,7927,7928,7929,7930,7931,7932,7933,7934,7935,
579 7936,7937,7938,7939,7940,7941,7942,7943,7944,7945,7946,7947,7948,7949,7950,7951,
580 7952,7953,7954,7955,7956,7957,7958,7959,7960,7961,7962,7963,7964,7965,7966,7967,
581 7968,7969,7970,7971,7972,7973,7974,7975,7976,7977,7978,7979,7980,7981,7982,7983,
582 7984,7985,7986,7987,7988,7989,7990,7991,7992,7993,7994,7995,7996,7997,7998,7999,
583 8000,8001,8002,8003,8004,8005,8006,8007,8008,8009,8010,8011,8012,8013,8014,8015,
584 8016,8017,8018,8019,8020,8021,8022,8023,8024,8025,8026,8027,8028,8029,8030,8031,
585 8032,8033,8034,8035,8036,8037,8038,8039,8040,8041,8042,8043,8044,8045,8046,8047,
586 8048,8049,8050,8051,8052,8053,8054,8055,8056,8057,8058,8059,8060,8061,8062,8063,
587 8064,8065,8066,8067,8068,8069,8070,8071,8072,8073,8074,8075,8076,8077,8078,8079,
588 8080,8081,8082,8083,8084,8085,8086,8087,8088,8089,8090,8091,8092,8093,8094,8095,
589 8096,8097,8098,8099,8100,8101,8102,8103,8104,8105,8106,8107,8108,8109,8110,8111,
590 8112,8113,8114,8115,8116,8117,8118,8119,8120,8121,8122,8123,8124,8125,8126,8127,
591 8128,8129,8130,8131,8132,8133,8134,8135,8136,8137,8138,8139,8140,8141,8142,8143,
592 8144,8145,8146,8147,8148,8149,8150,8151,8152,8153,8154,8155,8156,8157,8158,8159,
593 8160,8161,8162,8163,8164,8165,8166,8167,8168,8169,8170,8171,8172,8173,8174,8175,
594 8176,8177,8178,8179,8180,8181,8182,8183,8184,8185,8186,8187,8188,8189,8190,8191,
595 8192,8193,8194,8195,8196,8197,8198,8199,8200,8201,8202,8203,8204,8205,8206,8207,
596 8208,8209,8210,8211,8212,8213,8214,8215,8216,8217,8218,8219,8220,8221,8222,8223,
597 8224,8225,8226,8227,8228,8229,8230,8231,8232,8233,8234,8235,8236,8237,8238,8239,
598 8240,8241,8242,8243,8244,8245,8246,8247,8248,8249,8250,8251,8252,8253,8254,8255,
599 8256,8257,8258,8259,8260,8261,8262,8263,8264,8265,8266,8267,8268,8269,8270,8271,
600 8272,8273,8274,8275,8276,8277,8278,8279,8280,8281,8282,8283,8284,8285,8286,8287,
601 8288,8289,8290,8291,8292,8293,8294,8295,8296,8297,8298,8299,8300,8301,8302,8303,
602 8304,8305,8306,8307,8308,8309,8310,8311,8312,8313,8314,8315,8316,8317,8318,8319,
603 8320,8321,8322,8323,8324,8325,8326,8327,8328,8329,8330,8331,8332,8333,8334,8335,
604 8336,8337,8338,8339,8340,8341,8342,8343,8344,8345,8346,8347,8348,8349,8350,8351,
605 8352,8353,8354,8355,8356,8357,8358,8359,8360,8361,8362,8363,8364,8365,8366,8367,
606 8368,8369,8370,8371,8372,8373,8374,8375,8376,8377,8378,8379,8380,8381,8382,8383,
607 8384,8385,8386,8387,8388,8389,8390,8391,8392,8393,8394,8395,8396,8397,8398,8399,
608 8400,8401,8402,8403,8404,8405,8406,8407,8408,8409,8410,8411,8412,8413,8414,8415,
609 8416,8417,8418,8419,8420,8421,8422,8423,8424,8425,8426,8427,8428,8429,8430,8431,
610 8432,8433,8434,8435,8436,8437,8438,8439,8440,8441,8442,8443,8444,8445,8446,8447,
611 8448,8449,8450,8451,8452,8453,8454,8455,8456,8457,8458,8459,8460,8461,8462,8463,
612 8464,8465,8466,8467,8468,8469,8470,8471,8472,8473,8474,8475,8476,8477,8478,8479,
613 8480,8481,8482,8483,8484,8485,8486,8487,8488,8489,8490,8491,8492,8493,8494,8495,
614 8496,8497,8498,8499,8500,8501,8502,8503,8504,8505,8506,8507,8508,8509,8510,8511,
615 8512,8513,8514,8515,8516,8517,8518,8519,8520,8521,8522,8523,8524,8525,8526,8527,
616 8528,8529,8530,8531,8532,8533,8534,8535,8536,8537,8538,8539,8540,8541,8542,8543,
617 8544,8545,8546,8547,8548,8549,8550,8551,8552,8553,8554,8555,8556,8557,8558,8559,
618 8560,8561,8562,8563,8564,8565,8566,8567,8568,8569,8570,8571,8572,8573,8574,8575,
619 8576,8577,8578,8579,8580,8581,8582,8583,8584,8585,8586,8587,8588,8589,8590,8591,
620 8592,8593,8594,8595,8596,8597,8598,8599,8600,8601,8602,8603,8604,8605,8606,8607,
621 8608,8609,8610,8611,8612,8613,8614,8615,8616,8617,8618,8619,8620,8621,8622,8623,
622 8624,8625,8626,8627,8628,8629,8630,8631,8632,8633,8634,8635,8636,8637,8638,8639,
623 8640,8641,8642,8643,8644,8645,8646,8647,8648,8649,8650,8651,8652,8653,8654,8655,
624 8656,8657,8658,8659,8660,8661,8662,8663,8664,8665,8666,8667,8668,8669,8670,8671,
625 8672,8673,8674,8675,8676,8677,8678,8679,8680,8681,8682,8683,8684,8685,8686,8687,
626 8688,8689,8690,8691,8692,8693,8694,8695,8696,8697,8698,8699,8700,8701,8702,8703,
627 8704,8705,8706,8707,8708,8709,8710,8711,8712,8713,8714,8715,8716,8717,8718,8719,
628 8720,8721,8722,8723,8724,8725,8726,8727,8728,8729,8730,8731,8732,8733,8734,8735,
629 8736,8737,8738,8739,8740,8741
630 ****************************************************************************************/
631 };
632 }
+459
-0
src/main/java/org/mozilla/universalchardet/prober/distributionanalysis/EUCTWDistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public class EUCTWDistributionAnalysis extends CharDistributionAnalysis {
40 ////////////////////////////////////////////////////////////////
41 // constants
42 ////////////////////////////////////////////////////////////////
43 public static final float EUCTW_TYPICAL_DISTRIBUTION_RATIO = 0.75f;
44 public static final int HIGHBYTE_BEGIN = 0xC4;
45 public static final int HIGHBYTE_END = 0xFE;
46 public static final int LOWBYTE_BEGIN = 0xA1;
47 public static final int LOWBYTE_END = 0xFE;
48
49
50 ////////////////////////////////////////////////////////////////
51 // methods
52 ////////////////////////////////////////////////////////////////
53 public EUCTWDistributionAnalysis() {
54 super();
55 this.charToFreqOrder = euctwCharToFreqOrder;
56 this.typicalDistributionRatio = EUCTW_TYPICAL_DISTRIBUTION_RATIO;
57 }
58
59 @Override
60 protected int getOrder(final byte[] buf, int offset) {
61 int highbyte = buf[offset] & 0xFF;
62 if (highbyte >= HIGHBYTE_BEGIN) {
63 int lowbyte = buf[offset+1] & 0xFF;
64 return (94 * (highbyte - HIGHBYTE_BEGIN) + lowbyte - LOWBYTE_BEGIN);
65 } else {
66 return -1;
67 }
68 }
69
70
71 ////////////////////////////////////////////////////////////////
72 // constants continued
73 ////////////////////////////////////////////////////////////////
74 private static final int[] euctwCharToFreqOrder = new int[] {
75 1,1800,1506, 255,1431, 198, 9, 82, 6,7310, 177, 202,3615,1256,2808, 110, // 2742
76 3735, 33,3241, 261, 76, 44,2113, 16,2931,2184,1176, 659,3868, 26,3404,2643, // 2758
77 1198,3869,3313,4060, 410,2211, 302, 590, 361,1963, 8, 204, 58,4296,7311,1931, // 2774
78 63,7312,7313, 317,1614, 75, 222, 159,4061,2412,1480,7314,3500,3068, 224,2809, // 2790
79 3616, 3, 10,3870,1471, 29,2774,1135,2852,1939, 873, 130,3242,1123, 312,7315, // 2806
80 4297,2051, 507, 252, 682,7316, 142,1914, 124, 206,2932, 34,3501,3173, 64, 604, // 2822
81 7317,2494,1976,1977, 155,1990, 645, 641,1606,7318,3405, 337, 72, 406,7319, 80, // 2838
82 630, 238,3174,1509, 263, 939,1092,2644, 756,1440,1094,3406, 449, 69,2969, 591, // 2854
83 179,2095, 471, 115,2034,1843, 60, 50,2970, 134, 806,1868, 734,2035,3407, 180, // 2870
84 995,1607, 156, 537,2893, 688,7320, 319,1305, 779,2144, 514,2374, 298,4298, 359, // 2886
85 2495, 90,2707,1338, 663, 11, 906,1099,2545, 20,2436, 182, 532,1716,7321, 732, // 2902
86 1376,4062,1311,1420,3175, 25,2312,1056, 113, 399, 382,1949, 242,3408,2467, 529, // 2918
87 3243, 475,1447,3617,7322, 117, 21, 656, 810,1297,2295,2329,3502,7323, 126,4063, // 2934
88 706, 456, 150, 613,4299, 71,1118,2036,4064, 145,3069, 85, 835, 486,2114,1246, // 2950
89 1426, 428, 727,1285,1015, 800, 106, 623, 303,1281,7324,2127,2354, 347,3736, 221, // 2966
90 3503,3110,7325,1955,1153,4065, 83, 296,1199,3070, 192, 624, 93,7326, 822,1897, // 2982
91 2810,3111, 795,2064, 991,1554,1542,1592, 27, 43,2853, 859, 139,1456, 860,4300, // 2998
92 437, 712,3871, 164,2392,3112, 695, 211,3017,2096, 195,3872,1608,3504,3505,3618, // 3014
93 3873, 234, 811,2971,2097,3874,2229,1441,3506,1615,2375, 668,2076,1638, 305, 228, // 3030
94 1664,4301, 467, 415,7327, 262,2098,1593, 239, 108, 300, 200,1033, 512,1247,2077, // 3046
95 7328,7329,2173,3176,3619,2673, 593, 845,1062,3244, 88,1723,2037,3875,1950, 212, // 3062
96 266, 152, 149, 468,1898,4066,4302, 77, 187,7330,3018, 37, 5,2972,7331,3876, // 3078
97 7332,7333, 39,2517,4303,2894,3177,2078, 55, 148, 74,4304, 545, 483,1474,1029, // 3094
98 1665, 217,1869,1531,3113,1104,2645,4067, 24, 172,3507, 900,3877,3508,3509,4305, // 3110
99 32,1408,2811,1312, 329, 487,2355,2247,2708, 784,2674, 4,3019,3314,1427,1788, // 3126
100 188, 109, 499,7334,3620,1717,1789, 888,1217,3020,4306,7335,3510,7336,3315,1520, // 3142
101 3621,3878, 196,1034, 775,7337,7338, 929,1815, 249, 439, 38,7339,1063,7340, 794, // 3158
102 3879,1435,2296, 46, 178,3245,2065,7341,2376,7342, 214,1709,4307, 804, 35, 707, // 3174
103 324,3622,1601,2546, 140, 459,4068,7343,7344,1365, 839, 272, 978,2257,2572,3409, // 3190
104 2128,1363,3623,1423, 697, 100,3071, 48, 70,1231, 495,3114,2193,7345,1294,7346, // 3206
105 2079, 462, 586,1042,3246, 853, 256, 988, 185,2377,3410,1698, 434,1084,7347,3411, // 3222
106 314,2615,2775,4308,2330,2331, 569,2280, 637,1816,2518, 757,1162,1878,1616,3412, // 3238
107 287,1577,2115, 768,4309,1671,2854,3511,2519,1321,3737, 909,2413,7348,4069, 933, // 3254
108 3738,7349,2052,2356,1222,4310, 765,2414,1322, 786,4311,7350,1919,1462,1677,2895, // 3270
109 1699,7351,4312,1424,2437,3115,3624,2590,3316,1774,1940,3413,3880,4070, 309,1369, // 3286
110 1130,2812, 364,2230,1653,1299,3881,3512,3882,3883,2646, 525,1085,3021, 902,2000, // 3302
111 1475, 964,4313, 421,1844,1415,1057,2281, 940,1364,3116, 376,4314,4315,1381, 7, // 3318
112 2520, 983,2378, 336,1710,2675,1845, 321,3414, 559,1131,3022,2742,1808,1132,1313, // 3334
113 265,1481,1857,7352, 352,1203,2813,3247, 167,1089, 420,2814, 776, 792,1724,3513, // 3350
114 4071,2438,3248,7353,4072,7354, 446, 229, 333,2743, 901,3739,1200,1557,4316,2647, // 3366
115 1920, 395,2744,2676,3740,4073,1835, 125, 916,3178,2616,4317,7355,7356,3741,7357, // 3382
116 7358,7359,4318,3117,3625,1133,2547,1757,3415,1510,2313,1409,3514,7360,2145, 438, // 3398
117 2591,2896,2379,3317,1068, 958,3023, 461, 311,2855,2677,4074,1915,3179,4075,1978, // 3414
118 383, 750,2745,2617,4076, 274, 539, 385,1278,1442,7361,1154,1964, 384, 561, 210, // 3430
119 98,1295,2548,3515,7362,1711,2415,1482,3416,3884,2897,1257, 129,7363,3742, 642, // 3446
120 523,2776,2777,2648,7364, 141,2231,1333, 68, 176, 441, 876, 907,4077, 603,2592, // 3462
121 710, 171,3417, 404, 549, 18,3118,2393,1410,3626,1666,7365,3516,4319,2898,4320, // 3478
122 7366,2973, 368,7367, 146, 366, 99, 871,3627,1543, 748, 807,1586,1185, 22,2258, // 3494
123 379,3743,3180,7368,3181, 505,1941,2618,1991,1382,2314,7369, 380,2357, 218, 702, // 3510
124 1817,1248,3418,3024,3517,3318,3249,7370,2974,3628, 930,3250,3744,7371, 59,7372, // 3526
125 585, 601,4078, 497,3419,1112,1314,4321,1801,7373,1223,1472,2174,7374, 749,1836, // 3542
126 690,1899,3745,1772,3885,1476, 429,1043,1790,2232,2116, 917,4079, 447,1086,1629, // 3558
127 7375, 556,7376,7377,2020,1654, 844,1090, 105, 550, 966,1758,2815,1008,1782, 686, // 3574
128 1095,7378,2282, 793,1602,7379,3518,2593,4322,4080,2933,2297,4323,3746, 980,2496, // 3590
129 544, 353, 527,4324, 908,2678,2899,7380, 381,2619,1942,1348,7381,1341,1252, 560, // 3606
130 3072,7382,3420,2856,7383,2053, 973, 886,2080, 143,4325,7384,7385, 157,3886, 496, // 3622
131 4081, 57, 840, 540,2038,4326,4327,3421,2117,1445, 970,2259,1748,1965,2081,4082, // 3638
132 3119,1234,1775,3251,2816,3629, 773,1206,2129,1066,2039,1326,3887,1738,1725,4083, // 3654
133 279,3120, 51,1544,2594, 423,1578,2130,2066, 173,4328,1879,7386,7387,1583, 264, // 3670
134 610,3630,4329,2439, 280, 154,7388,7389,7390,1739, 338,1282,3073, 693,2857,1411, // 3686
135 1074,3747,2440,7391,4330,7392,7393,1240, 952,2394,7394,2900,1538,2679, 685,1483, // 3702
136 4084,2468,1436, 953,4085,2054,4331, 671,2395, 79,4086,2441,3252, 608, 567,2680, // 3718
137 3422,4087,4088,1691, 393,1261,1791,2396,7395,4332,7396,7397,7398,7399,1383,1672, // 3734
138 3748,3182,1464, 522,1119, 661,1150, 216, 675,4333,3888,1432,3519, 609,4334,2681, // 3750
139 2397,7400,7401,7402,4089,3025, 0,7403,2469, 315, 231,2442, 301,3319,4335,2380, // 3766
140 7404, 233,4090,3631,1818,4336,4337,7405, 96,1776,1315,2082,7406, 257,7407,1809, // 3782
141 3632,2709,1139,1819,4091,2021,1124,2163,2778,1777,2649,7408,3074, 363,1655,3183, // 3798
142 7409,2975,7410,7411,7412,3889,1567,3890, 718, 103,3184, 849,1443, 341,3320,2934, // 3814
143 1484,7413,1712, 127, 67, 339,4092,2398, 679,1412, 821,7414,7415, 834, 738, 351, // 3830
144 2976,2146, 846, 235,1497,1880, 418,1992,3749,2710, 186,1100,2147,2746,3520,1545, // 3846
145 1355,2935,2858,1377, 583,3891,4093,2573,2977,7416,1298,3633,1078,2549,3634,2358, // 3862
146 78,3750,3751, 267,1289,2099,2001,1594,4094, 348, 369,1274,2194,2175,1837,4338, // 3878
147 1820,2817,3635,2747,2283,2002,4339,2936,2748, 144,3321, 882,4340,3892,2749,3423, // 3894
148 4341,2901,7417,4095,1726, 320,7418,3893,3026, 788,2978,7419,2818,1773,1327,2859, // 3910
149 3894,2819,7420,1306,4342,2003,1700,3752,3521,2359,2650, 787,2022, 506, 824,3636, // 3926
150 534, 323,4343,1044,3322,2023,1900, 946,3424,7421,1778,1500,1678,7422,1881,4344, // 3942
151 165, 243,4345,3637,2521, 123, 683,4096, 764,4346, 36,3895,1792, 589,2902, 816, // 3958
152 626,1667,3027,2233,1639,1555,1622,3753,3896,7423,3897,2860,1370,1228,1932, 891, // 3974
153 2083,2903, 304,4097,7424, 292,2979,2711,3522, 691,2100,4098,1115,4347, 118, 662, // 3990
154 7425, 611,1156, 854,2381,1316,2861, 2, 386, 515,2904,7426,7427,3253, 868,2234, // 4006
155 1486, 855,2651, 785,2212,3028,7428,1040,3185,3523,7429,3121, 448,7430,1525,7431, // 4022
156 2164,4348,7432,3754,7433,4099,2820,3524,3122, 503, 818,3898,3123,1568, 814, 676, // 4038
157 1444, 306,1749,7434,3755,1416,1030, 197,1428, 805,2821,1501,4349,7435,7436,7437, // 4054
158 1993,7438,4350,7439,7440,2195, 13,2779,3638,2980,3124,1229,1916,7441,3756,2131, // 4070
159 7442,4100,4351,2399,3525,7443,2213,1511,1727,1120,7444,7445, 646,3757,2443, 307, // 4086
160 7446,7447,1595,3186,7448,7449,7450,3639,1113,1356,3899,1465,2522,2523,7451, 519, // 4102
161 7452, 128,2132, 92,2284,1979,7453,3900,1512, 342,3125,2196,7454,2780,2214,1980, // 4118
162 3323,7455, 290,1656,1317, 789, 827,2360,7456,3758,4352, 562, 581,3901,7457, 401, // 4134
163 4353,2248, 94,4354,1399,2781,7458,1463,2024,4355,3187,1943,7459, 828,1105,4101, // 4150
164 1262,1394,7460,4102, 605,4356,7461,1783,2862,7462,2822, 819,2101, 578,2197,2937, // 4166
165 7463,1502, 436,3254,4103,3255,2823,3902,2905,3425,3426,7464,2712,2315,7465,7466, // 4182
166 2332,2067, 23,4357, 193, 826,3759,2102, 699,1630,4104,3075, 390,1793,1064,3526, // 4198
167 7467,1579,3076,3077,1400,7468,4105,1838,1640,2863,7469,4358,4359, 137,4106, 598, // 4214
168 3078,1966, 780, 104, 974,2938,7470, 278, 899, 253, 402, 572, 504, 493,1339,7471, // 4230
169 3903,1275,4360,2574,2550,7472,3640,3029,3079,2249, 565,1334,2713, 863, 41,7473, // 4246
170 7474,4361,7475,1657,2333, 19, 463,2750,4107, 606,7476,2981,3256,1087,2084,1323, // 4262
171 2652,2982,7477,1631,1623,1750,4108,2682,7478,2864, 791,2714,2653,2334, 232,2416, // 4278
172 7479,2983,1498,7480,2654,2620, 755,1366,3641,3257,3126,2025,1609, 119,1917,3427, // 4294
173 862,1026,4109,7481,3904,3760,4362,3905,4363,2260,1951,2470,7482,1125, 817,4110, // 4310
174 4111,3906,1513,1766,2040,1487,4112,3030,3258,2824,3761,3127,7483,7484,1507,7485, // 4326
175 2683, 733, 40,1632,1106,2865, 345,4113, 841,2524, 230,4364,2984,1846,3259,3428, // 4342
176 7486,1263, 986,3429,7487, 735, 879, 254,1137, 857, 622,1300,1180,1388,1562,3907, // 4358
177 3908,2939, 967,2751,2655,1349, 592,2133,1692,3324,2985,1994,4114,1679,3909,1901, // 4374
178 2185,7488, 739,3642,2715,1296,1290,7489,4115,2198,2199,1921,1563,2595,2551,1870, // 4390
179 2752,2986,7490, 435,7491, 343,1108, 596, 17,1751,4365,2235,3430,3643,7492,4366, // 4406
180 294,3527,2940,1693, 477, 979, 281,2041,3528, 643,2042,3644,2621,2782,2261,1031, // 4422
181 2335,2134,2298,3529,4367, 367,1249,2552,7493,3530,7494,4368,1283,3325,2004, 240, // 4438
182 1762,3326,4369,4370, 836,1069,3128, 474,7495,2148,2525, 268,3531,7496,3188,1521, // 4454
183 1284,7497,1658,1546,4116,7498,3532,3533,7499,4117,3327,2684,1685,4118, 961,1673, // 4470
184 2622, 190,2005,2200,3762,4371,4372,7500, 570,2497,3645,1490,7501,4373,2623,3260, // 4486
185 1956,4374, 584,1514, 396,1045,1944,7502,4375,1967,2444,7503,7504,4376,3910, 619, // 4502
186 7505,3129,3261, 215,2006,2783,2553,3189,4377,3190,4378, 763,4119,3763,4379,7506, // 4518
187 7507,1957,1767,2941,3328,3646,1174, 452,1477,4380,3329,3130,7508,2825,1253,2382, // 4534
188 2186,1091,2285,4120, 492,7509, 638,1169,1824,2135,1752,3911, 648, 926,1021,1324, // 4550
189 4381, 520,4382, 997, 847,1007, 892,4383,3764,2262,1871,3647,7510,2400,1784,4384, // 4566
190 1952,2942,3080,3191,1728,4121,2043,3648,4385,2007,1701,3131,1551, 30,2263,4122, // 4582
191 7511,2026,4386,3534,7512, 501,7513,4123, 594,3431,2165,1821,3535,3432,3536,3192, // 4598
192 829,2826,4124,7514,1680,3132,1225,4125,7515,3262,4387,4126,3133,2336,7516,4388, // 4614
193 4127,7517,3912,3913,7518,1847,2383,2596,3330,7519,4389, 374,3914, 652,4128,4129, // 4630
194 375,1140, 798,7520,7521,7522,2361,4390,2264, 546,1659, 138,3031,2445,4391,7523, // 4646
195 2250, 612,1848, 910, 796,3765,1740,1371, 825,3766,3767,7524,2906,2554,7525, 692, // 4662
196 444,3032,2624, 801,4392,4130,7526,1491, 244,1053,3033,4131,4132, 340,7527,3915, // 4678
197 1041,2987, 293,1168, 87,1357,7528,1539, 959,7529,2236, 721, 694,4133,3768, 219, // 4694
198 1478, 644,1417,3331,2656,1413,1401,1335,1389,3916,7530,7531,2988,2362,3134,1825, // 4710
199 730,1515, 184,2827, 66,4393,7532,1660,2943, 246,3332, 378,1457, 226,3433, 975, // 4726
200 3917,2944,1264,3537, 674, 696,7533, 163,7534,1141,2417,2166, 713,3538,3333,4394, // 4742
201 3918,7535,7536,1186, 15,7537,1079,1070,7538,1522,3193,3539, 276,1050,2716, 758, // 4758
202 1126, 653,2945,3263,7539,2337, 889,3540,3919,3081,2989, 903,1250,4395,3920,3434, // 4774
203 3541,1342,1681,1718, 766,3264, 286, 89,2946,3649,7540,1713,7541,2597,3334,2990, // 4790
204 7542,2947,2215,3194,2866,7543,4396,2498,2526, 181, 387,1075,3921, 731,2187,3335, // 4806
205 7544,3265, 310, 313,3435,2299, 770,4134, 54,3034, 189,4397,3082,3769,3922,7545, // 4822
206 1230,1617,1849, 355,3542,4135,4398,3336, 111,4136,3650,1350,3135,3436,3035,4137, // 4838
207 2149,3266,3543,7546,2784,3923,3924,2991, 722,2008,7547,1071, 247,1207,2338,2471, // 4854
208 1378,4399,2009, 864,1437,1214,4400, 373,3770,1142,2216, 667,4401, 442,2753,2555, // 4870
209 3771,3925,1968,4138,3267,1839, 837, 170,1107, 934,1336,1882,7548,7549,2118,4139, // 4886
210 2828, 743,1569,7550,4402,4140, 582,2384,1418,3437,7551,1802,7552, 357,1395,1729, // 4902
211 3651,3268,2418,1564,2237,7553,3083,3772,1633,4403,1114,2085,4141,1532,7554, 482, // 4918
212 2446,4404,7555,7556,1492, 833,1466,7557,2717,3544,1641,2829,7558,1526,1272,3652, // 4934
213 4142,1686,1794, 416,2556,1902,1953,1803,7559,3773,2785,3774,1159,2316,7560,2867, // 4950
214 4405,1610,1584,3036,2419,2754, 443,3269,1163,3136,7561,7562,3926,7563,4143,2499, // 4966
215 3037,4406,3927,3137,2103,1647,3545,2010,1872,4144,7564,4145, 431,3438,7565, 250, // 4982
216 97, 81,4146,7566,1648,1850,1558, 160, 848,7567, 866, 740,1694,7568,2201,2830, // 4998
217 3195,4147,4407,3653,1687, 950,2472, 426, 469,3196,3654,3655,3928,7569,7570,1188, // 5014
218 424,1995, 861,3546,4148,3775,2202,2685, 168,1235,3547,4149,7571,2086,1674,4408, // 5030
219 3337,3270, 220,2557,1009,7572,3776, 670,2992, 332,1208, 717,7573,7574,3548,2447, // 5046
220 3929,3338,7575, 513,7576,1209,2868,3339,3138,4409,1080,7577,7578,7579,7580,2527, // 5062
221 3656,3549, 815,1587,3930,3931,7581,3550,3439,3777,1254,4410,1328,3038,1390,3932, // 5078
222 1741,3933,3778,3934,7582, 236,3779,2448,3271,7583,7584,3657,3780,1273,3781,4411, // 5094
223 7585, 308,7586,4412, 245,4413,1851,2473,1307,2575, 430, 715,2136,2449,7587, 270, // 5110
224 199,2869,3935,7588,3551,2718,1753, 761,1754, 725,1661,1840,4414,3440,3658,7589, // 5126
225 7590, 587, 14,3272, 227,2598, 326, 480,2265, 943,2755,3552, 291, 650,1883,7591, // 5142
226 1702,1226, 102,1547, 62,3441, 904,4415,3442,1164,4150,7592,7593,1224,1548,2756, // 5158
227 391, 498,1493,7594,1386,1419,7595,2055,1177,4416, 813, 880,1081,2363, 566,1145, // 5174
228 4417,2286,1001,1035,2558,2599,2238, 394,1286,7596,7597,2068,7598, 86,1494,1730, // 5190
229 3936, 491,1588, 745, 897,2948, 843,3340,3937,2757,2870,3273,1768, 998,2217,2069, // 5206
230 397,1826,1195,1969,3659,2993,3341, 284,7599,3782,2500,2137,2119,1903,7600,3938, // 5222
231 2150,3939,4151,1036,3443,1904, 114,2559,4152, 209,1527,7601,7602,2949,2831,2625, // 5238
232 2385,2719,3139, 812,2560,7603,3274,7604,1559, 737,1884,3660,1210, 885, 28,2686, // 5254
233 3553,3783,7605,4153,1004,1779,4418,7606, 346,1981,2218,2687,4419,3784,1742, 797, // 5270
234 1642,3940,1933,1072,1384,2151, 896,3941,3275,3661,3197,2871,3554,7607,2561,1958, // 5286
235 4420,2450,1785,7608,7609,7610,3942,4154,1005,1308,3662,4155,2720,4421,4422,1528, // 5302
236 2600, 161,1178,4156,1982, 987,4423,1101,4157, 631,3943,1157,3198,2420,1343,1241, // 5318
237 1016,2239,2562, 372, 877,2339,2501,1160, 555,1934, 911,3944,7611, 466,1170, 169, // 5334
238 1051,2907,2688,3663,2474,2994,1182,2011,2563,1251,2626,7612, 992,2340,3444,1540, // 5350
239 2721,1201,2070,2401,1996,2475,7613,4424, 528,1922,2188,1503,1873,1570,2364,3342, // 5366
240 3276,7614, 557,1073,7615,1827,3445,2087,2266,3140,3039,3084, 767,3085,2786,4425, // 5382
241 1006,4158,4426,2341,1267,2176,3664,3199, 778,3945,3200,2722,1597,2657,7616,4427, // 5398
242 7617,3446,7618,7619,7620,3277,2689,1433,3278, 131, 95,1504,3946, 723,4159,3141, // 5414
243 1841,3555,2758,2189,3947,2027,2104,3665,7621,2995,3948,1218,7622,3343,3201,3949, // 5430
244 4160,2576, 248,1634,3785, 912,7623,2832,3666,3040,3786, 654, 53,7624,2996,7625, // 5446
245 1688,4428, 777,3447,1032,3950,1425,7626, 191, 820,2120,2833, 971,4429, 931,3202, // 5462
246 135, 664, 783,3787,1997, 772,2908,1935,3951,3788,4430,2909,3203, 282,2723, 640, // 5478
247 1372,3448,1127, 922, 325,3344,7627,7628, 711,2044,7629,7630,3952,2219,2787,1936, // 5494
248 3953,3345,2220,2251,3789,2300,7631,4431,3790,1258,3279,3954,3204,2138,2950,3955, // 5510
249 3956,7632,2221, 258,3205,4432, 101,1227,7633,3280,1755,7634,1391,3281,7635,2910, // 5526
250 2056, 893,7636,7637,7638,1402,4161,2342,7639,7640,3206,3556,7641,7642, 878,1325, // 5542
251 1780,2788,4433, 259,1385,2577, 744,1183,2267,4434,7643,3957,2502,7644, 684,1024, // 5558
252 4162,7645, 472,3557,3449,1165,3282,3958,3959, 322,2152, 881, 455,1695,1152,1340, // 5574
253 660, 554,2153,4435,1058,4436,4163, 830,1065,3346,3960,4437,1923,7646,1703,1918, // 5590
254 7647, 932,2268, 122,7648,4438, 947, 677,7649,3791,2627, 297,1905,1924,2269,4439, // 5606
255 2317,3283,7650,7651,4164,7652,4165, 84,4166, 112, 989,7653, 547,1059,3961, 701, // 5622
256 3558,1019,7654,4167,7655,3450, 942, 639, 457,2301,2451, 993,2951, 407, 851, 494, // 5638
257 4440,3347, 927,7656,1237,7657,2421,3348, 573,4168, 680, 921,2911,1279,1874, 285, // 5654
258 790,1448,1983, 719,2167,7658,7659,4441,3962,3963,1649,7660,1541, 563,7661,1077, // 5670
259 7662,3349,3041,3451, 511,2997,3964,3965,3667,3966,1268,2564,3350,3207,4442,4443, // 5686
260 7663, 535,1048,1276,1189,2912,2028,3142,1438,1373,2834,2952,1134,2012,7664,4169, // 5702
261 1238,2578,3086,1259,7665, 700,7666,2953,3143,3668,4170,7667,4171,1146,1875,1906, // 5718
262 4444,2601,3967, 781,2422, 132,1589, 203, 147, 273,2789,2402, 898,1786,2154,3968, // 5734
263 3969,7668,3792,2790,7669,7670,4445,4446,7671,3208,7672,1635,3793, 965,7673,1804, // 5750
264 2690,1516,3559,1121,1082,1329,3284,3970,1449,3794, 65,1128,2835,2913,2759,1590, // 5766
265 3795,7674,7675, 12,2658, 45, 976,2579,3144,4447, 517,2528,1013,1037,3209,7676, // 5782
266 3796,2836,7677,3797,7678,3452,7679,2602, 614,1998,2318,3798,3087,2724,2628,7680, // 5798
267 2580,4172, 599,1269,7681,1810,3669,7682,2691,3088, 759,1060, 489,1805,3351,3285, // 5814
268 1358,7683,7684,2386,1387,1215,2629,2252, 490,7685,7686,4173,1759,2387,2343,7687, // 5830
269 4448,3799,1907,3971,2630,1806,3210,4449,3453,3286,2760,2344, 874,7688,7689,3454, // 5846
270 3670,1858, 91,2914,3671,3042,3800,4450,7690,3145,3972,2659,7691,3455,1202,1403, // 5862
271 3801,2954,2529,1517,2503,4451,3456,2504,7692,4452,7693,2692,1885,1495,1731,3973, // 5878
272 2365,4453,7694,2029,7695,7696,3974,2693,1216, 237,2581,4174,2319,3975,3802,4454, // 5894
273 4455,2694,3560,3457, 445,4456,7697,7698,7699,7700,2761, 61,3976,3672,1822,3977, // 5910
274 7701, 687,2045, 935, 925, 405,2660, 703,1096,1859,2725,4457,3978,1876,1367,2695, // 5926
275 3352, 918,2105,1781,2476, 334,3287,1611,1093,4458, 564,3146,3458,3673,3353, 945, // 5942
276 2631,2057,4459,7702,1925, 872,4175,7703,3459,2696,3089, 349,4176,3674,3979,4460, // 5958
277 3803,4177,3675,2155,3980,4461,4462,4178,4463,2403,2046, 782,3981, 400, 251,4179, // 5974
278 1624,7704,7705, 277,3676, 299,1265, 476,1191,3804,2121,4180,4181,1109, 205,7706, // 5990
279 2582,1000,2156,3561,1860,7707,7708,7709,4464,7710,4465,2565, 107,2477,2157,3982, // 6006
280 3460,3147,7711,1533, 541,1301, 158, 753,4182,2872,3562,7712,1696, 370,1088,4183, // 6022
281 4466,3563, 579, 327, 440, 162,2240, 269,1937,1374,3461, 968,3043, 56,1396,3090, // 6038
282 2106,3288,3354,7713,1926,2158,4467,2998,7714,3564,7715,7716,3677,4468,2478,7717, // 6054
283 2791,7718,1650,4469,7719,2603,7720,7721,3983,2661,3355,1149,3356,3984,3805,3985, // 6070
284 7722,1076, 49,7723, 951,3211,3289,3290, 450,2837, 920,7724,1811,2792,2366,4184, // 6086
285 1908,1138,2367,3806,3462,7725,3212,4470,1909,1147,1518,2423,4471,3807,7726,4472, // 6102
286 2388,2604, 260,1795,3213,7727,7728,3808,3291, 708,7729,3565,1704,7730,3566,1351, // 6118
287 1618,3357,2999,1886, 944,4185,3358,4186,3044,3359,4187,7731,3678, 422, 413,1714, // 6134
288 3292, 500,2058,2345,4188,2479,7732,1344,1910, 954,7733,1668,7734,7735,3986,2404, // 6150
289 4189,3567,3809,4190,7736,2302,1318,2505,3091, 133,3092,2873,4473, 629, 31,2838, // 6166
290 2697,3810,4474, 850, 949,4475,3987,2955,1732,2088,4191,1496,1852,7737,3988, 620, // 6182
291 3214, 981,1242,3679,3360,1619,3680,1643,3293,2139,2452,1970,1719,3463,2168,7738, // 6198
292 3215,7739,7740,3361,1828,7741,1277,4476,1565,2047,7742,1636,3568,3093,7743, 869, // 6214
293 2839, 655,3811,3812,3094,3989,3000,3813,1310,3569,4477,7744,7745,7746,1733, 558, // 6230
294 4478,3681, 335,1549,3045,1756,4192,3682,1945,3464,1829,1291,1192, 470,2726,2107, // 6246
295 2793, 913,1054,3990,7747,1027,7748,3046,3991,4479, 982,2662,3362,3148,3465,3216, // 6262
296 3217,1946,2794,7749, 571,4480,7750,1830,7751,3570,2583,1523,2424,7752,2089, 984, // 6278
297 4481,3683,1959,7753,3684, 852, 923,2795,3466,3685, 969,1519, 999,2048,2320,1705, // 6294
298 7754,3095, 615,1662, 151, 597,3992,2405,2321,1049, 275,4482,3686,4193, 568,3687, // 6310
299 3571,2480,4194,3688,7755,2425,2270, 409,3218,7756,1566,2874,3467,1002, 769,2840, // 6326
300 194,2090,3149,3689,2222,3294,4195, 628,1505,7757,7758,1763,2177,3001,3993, 521, // 6342
301 1161,2584,1787,2203,2406,4483,3994,1625,4196,4197, 412, 42,3096, 464,7759,2632, // 6358
302 4484,3363,1760,1571,2875,3468,2530,1219,2204,3814,2633,2140,2368,4485,4486,3295, // 6374
303 1651,3364,3572,7760,7761,3573,2481,3469,7762,3690,7763,7764,2271,2091, 460,7765, // 6390
304 4487,7766,3002, 962, 588,3574, 289,3219,2634,1116, 52,7767,3047,1796,7768,7769, // 6406
305 7770,1467,7771,1598,1143,3691,4198,1984,1734,1067,4488,1280,3365, 465,4489,1572, // 6422
306 510,7772,1927,2241,1812,1644,3575,7773,4490,3692,7774,7775,2663,1573,1534,7776, // 6438
307 7777,4199, 536,1807,1761,3470,3815,3150,2635,7778,7779,7780,4491,3471,2915,1911, // 6454
308 2796,7781,3296,1122, 377,3220,7782, 360,7783,7784,4200,1529, 551,7785,2059,3693, // 6470
309 1769,2426,7786,2916,4201,3297,3097,2322,2108,2030,4492,1404, 136,1468,1479, 672, // 6486
310 1171,3221,2303, 271,3151,7787,2762,7788,2049, 678,2727, 865,1947,4493,7789,2013, // 6502
311 3995,2956,7790,2728,2223,1397,3048,3694,4494,4495,1735,2917,3366,3576,7791,3816, // 6518
312 509,2841,2453,2876,3817,7792,7793,3152,3153,4496,4202,2531,4497,2304,1166,1010, // 6534
313 552, 681,1887,7794,7795,2957,2958,3996,1287,1596,1861,3154, 358, 453, 736, 175, // 6550
314 478,1117, 905,1167,1097,7796,1853,1530,7797,1706,7798,2178,3472,2287,3695,3473, // 6566
315 3577,4203,2092,4204,7799,3367,1193,2482,4205,1458,2190,2205,1862,1888,1421,3298, // 6582
316 2918,3049,2179,3474, 595,2122,7800,3997,7801,7802,4206,1707,2636, 223,3696,1359, // 6598
317 751,3098, 183,3475,7803,2797,3003, 419,2369, 633, 704,3818,2389, 241,7804,7805, // 6614
318 7806, 838,3004,3697,2272,2763,2454,3819,1938,2050,3998,1309,3099,2242,1181,7807, // 6630
319 1136,2206,3820,2370,1446,4207,2305,4498,7808,7809,4208,1055,2605, 484,3698,7810, // 6646
320 3999, 625,4209,2273,3368,1499,4210,4000,7811,4001,4211,3222,2274,2275,3476,7812, // 6662
321 7813,2764, 808,2606,3699,3369,4002,4212,3100,2532, 526,3370,3821,4213, 955,7814, // 6678
322 1620,4214,2637,2427,7815,1429,3700,1669,1831, 994, 928,7816,3578,1260,7817,7818, // 6694
323 7819,1948,2288, 741,2919,1626,4215,2729,2455, 867,1184, 362,3371,1392,7820,7821, // 6710
324 4003,4216,1770,1736,3223,2920,4499,4500,1928,2698,1459,1158,7822,3050,3372,2877, // 6726
325 1292,1929,2506,2842,3701,1985,1187,2071,2014,2607,4217,7823,2566,2507,2169,3702, // 6742
326 2483,3299,7824,3703,4501,7825,7826, 666,1003,3005,1022,3579,4218,7827,4502,1813, // 6758
327 2253, 574,3822,1603, 295,1535, 705,3823,4219, 283, 858, 417,7828,7829,3224,4503, // 6774
328 4504,3051,1220,1889,1046,2276,2456,4004,1393,1599, 689,2567, 388,4220,7830,2484, // 6790
329 802,7831,2798,3824,2060,1405,2254,7832,4505,3825,2109,1052,1345,3225,1585,7833, // 6806
330 809,7834,7835,7836, 575,2730,3477, 956,1552,1469,1144,2323,7837,2324,1560,2457, // 6822
331 3580,3226,4005, 616,2207,3155,2180,2289,7838,1832,7839,3478,4506,7840,1319,3704, // 6838
332 3705,1211,3581,1023,3227,1293,2799,7841,7842,7843,3826, 607,2306,3827, 762,2878, // 6854
333 1439,4221,1360,7844,1485,3052,7845,4507,1038,4222,1450,2061,2638,4223,1379,4508, // 6870
334 2585,7846,7847,4224,1352,1414,2325,2921,1172,7848,7849,3828,3829,7850,1797,1451, // 6886
335 7851,7852,7853,7854,2922,4006,4007,2485,2346, 411,4008,4009,3582,3300,3101,4509, // 6902
336 1561,2664,1452,4010,1375,7855,7856, 47,2959, 316,7857,1406,1591,2923,3156,7858, // 6918
337 1025,2141,3102,3157, 354,2731, 884,2224,4225,2407, 508,3706, 726,3583, 996,2428, // 6934
338 3584, 729,7859, 392,2191,1453,4011,4510,3707,7860,7861,2458,3585,2608,1675,2800, // 6950
339 919,2347,2960,2348,1270,4511,4012, 73,7862,7863, 647,7864,3228,2843,2255,1550, // 6966
340 1346,3006,7865,1332, 883,3479,7866,7867,7868,7869,3301,2765,7870,1212, 831,1347, // 6982
341 4226,4512,2326,3830,1863,3053, 720,3831,4513,4514,3832,7871,4227,7872,7873,4515, // 6998
342 7874,7875,1798,4516,3708,2609,4517,3586,1645,2371,7876,7877,2924, 669,2208,2665, // 7014
343 2429,7878,2879,7879,7880,1028,3229,7881,4228,2408,7882,2256,1353,7883,7884,4518, // 7030
344 3158, 518,7885,4013,7886,4229,1960,7887,2142,4230,7888,7889,3007,2349,2350,3833, // 7046
345 516,1833,1454,4014,2699,4231,4519,2225,2610,1971,1129,3587,7890,2766,7891,2961, // 7062
346 1422, 577,1470,3008,1524,3373,7892,7893, 432,4232,3054,3480,7894,2586,1455,2508, // 7078
347 2226,1972,1175,7895,1020,2732,4015,3481,4520,7896,2733,7897,1743,1361,3055,3482, // 7094
348 2639,4016,4233,4521,2290, 895, 924,4234,2170, 331,2243,3056, 166,1627,3057,1098, // 7110
349 7898,1232,2880,2227,3374,4522, 657, 403,1196,2372, 542,3709,3375,1600,4235,3483, // 7126
350 7899,4523,2767,3230, 576, 530,1362,7900,4524,2533,2666,3710,4017,7901, 842,3834, // 7142
351 7902,2801,2031,1014,4018, 213,2700,3376, 665, 621,4236,7903,3711,2925,2430,7904, // 7158
352 2431,3302,3588,3377,7905,4237,2534,4238,4525,3589,1682,4239,3484,1380,7906, 724, // 7174
353 2277, 600,1670,7907,1337,1233,4526,3103,2244,7908,1621,4527,7909, 651,4240,7910, // 7190
354 1612,4241,2611,7911,2844,7912,2734,2307,3058,7913, 716,2459,3059, 174,1255,2701, // 7206
355 4019,3590, 548,1320,1398, 728,4020,1574,7914,1890,1197,3060,4021,7915,3061,3062, // 7222
356 3712,3591,3713, 747,7916, 635,4242,4528,7917,7918,7919,4243,7920,7921,4529,7922, // 7238
357 3378,4530,2432, 451,7923,3714,2535,2072,4244,2735,4245,4022,7924,1764,4531,7925, // 7254
358 4246, 350,7926,2278,2390,2486,7927,4247,4023,2245,1434,4024, 488,4532, 458,4248, // 7270
359 4025,3715, 771,1330,2391,3835,2568,3159,2159,2409,1553,2667,3160,4249,7928,2487, // 7286
360 2881,2612,1720,2702,4250,3379,4533,7929,2536,4251,7930,3231,4252,2768,7931,2015, // 7302
361 2736,7932,1155,1017,3716,3836,7933,3303,2308, 201,1864,4253,1430,7934,4026,7935, // 7318
362 7936,7937,7938,7939,4254,1604,7940, 414,1865, 371,2587,4534,4535,3485,2016,3104, // 7334
363 4536,1708, 960,4255, 887, 389,2171,1536,1663,1721,7941,2228,4027,2351,2926,1580, // 7350
364 7942,7943,7944,1744,7945,2537,4537,4538,7946,4539,7947,2073,7948,7949,3592,3380, // 7366
365 2882,4256,7950,4257,2640,3381,2802, 673,2703,2460, 709,3486,4028,3593,4258,7951, // 7382
366 1148, 502, 634,7952,7953,1204,4540,3594,1575,4541,2613,3717,7954,3718,3105, 948, // 7398
367 3232, 121,1745,3837,1110,7955,4259,3063,2509,3009,4029,3719,1151,1771,3838,1488, // 7414
368 4030,1986,7956,2433,3487,7957,7958,2093,7959,4260,3839,1213,1407,2803, 531,2737, // 7430
369 2538,3233,1011,1537,7960,2769,4261,3106,1061,7961,3720,3721,1866,2883,7962,2017, // 7446
370 120,4262,4263,2062,3595,3234,2309,3840,2668,3382,1954,4542,7963,7964,3488,1047, // 7462
371 2704,1266,7965,1368,4543,2845, 649,3383,3841,2539,2738,1102,2846,2669,7966,7967, // 7478
372 1999,7968,1111,3596,2962,7969,2488,3842,3597,2804,1854,3384,3722,7970,7971,3385, // 7494
373 2410,2884,3304,3235,3598,7972,2569,7973,3599,2805,4031,1460, 856,7974,3600,7975, // 7510
374 2885,2963,7976,2886,3843,7977,4264, 632,2510, 875,3844,1697,3845,2291,7978,7979, // 7526
375 4544,3010,1239, 580,4545,4265,7980, 914, 936,2074,1190,4032,1039,2123,7981,7982, // 7542
376 7983,3386,1473,7984,1354,4266,3846,7985,2172,3064,4033, 915,3305,4267,4268,3306, // 7558
377 1605,1834,7986,2739, 398,3601,4269,3847,4034, 328,1912,2847,4035,3848,1331,4270, // 7574
378 3011, 937,4271,7987,3602,4036,4037,3387,2160,4546,3388, 524, 742, 538,3065,1012, // 7590
379 7988,7989,3849,2461,7990, 658,1103, 225,3850,7991,7992,4547,7993,4548,7994,3236, // 7606
380 1243,7995,4038, 963,2246,4549,7996,2705,3603,3161,7997,7998,2588,2327,7999,4550, // 7622
381 8000,8001,8002,3489,3307, 957,3389,2540,2032,1930,2927,2462, 870,2018,3604,1746, // 7638
382 2770,2771,2434,2463,8003,3851,8004,3723,3107,3724,3490,3390,3725,8005,1179,3066, // 7654
383 8006,3162,2373,4272,3726,2541,3163,3108,2740,4039,8007,3391,1556,2542,2292, 977, // 7670
384 2887,2033,4040,1205,3392,8008,1765,3393,3164,2124,1271,1689, 714,4551,3491,8009, // 7686
385 2328,3852, 533,4273,3605,2181, 617,8010,2464,3308,3492,2310,8011,8012,3165,8013, // 7702
386 8014,3853,1987, 618, 427,2641,3493,3394,8015,8016,1244,1690,8017,2806,4274,4552, // 7718
387 8018,3494,8019,8020,2279,1576, 473,3606,4275,3395, 972,8021,3607,8022,3067,8023, // 7734
388 8024,4553,4554,8025,3727,4041,4042,8026, 153,4555, 356,8027,1891,2888,4276,2143, // 7750
389 408, 803,2352,8028,3854,8029,4277,1646,2570,2511,4556,4557,3855,8030,3856,4278, // 7766
390 8031,2411,3396, 752,8032,8033,1961,2964,8034, 746,3012,2465,8035,4279,3728, 698, // 7782
391 4558,1892,4280,3608,2543,4559,3609,3857,8036,3166,3397,8037,1823,1302,4043,2706, // 7798
392 3858,1973,4281,8038,4282,3167, 823,1303,1288,1236,2848,3495,4044,3398, 774,3859, // 7814
393 8039,1581,4560,1304,2849,3860,4561,8040,2435,2161,1083,3237,4283,4045,4284, 344, // 7830
394 1173, 288,2311, 454,1683,8041,8042,1461,4562,4046,2589,8043,8044,4563, 985, 894, // 7846
395 8045,3399,3168,8046,1913,2928,3729,1988,8047,2110,1974,8048,4047,8049,2571,1194, // 7862
396 425,8050,4564,3169,1245,3730,4285,8051,8052,2850,8053, 636,4565,1855,3861, 760, // 7878
397 1799,8054,4286,2209,1508,4566,4048,1893,1684,2293,8055,8056,8057,4287,4288,2210, // 7894
398 479,8058,8059, 832,8060,4049,2489,8061,2965,2490,3731, 990,3109, 627,1814,2642, // 7910
399 4289,1582,4290,2125,2111,3496,4567,8062, 799,4291,3170,8063,4568,2112,1737,3013, // 7926
400 1018, 543, 754,4292,3309,1676,4569,4570,4050,8064,1489,8065,3497,8066,2614,2889, // 7942
401 4051,8067,8068,2966,8069,8070,8071,8072,3171,4571,4572,2182,1722,8073,3238,3239, // 7958
402 1842,3610,1715, 481, 365,1975,1856,8074,8075,1962,2491,4573,8076,2126,3611,3240, // 7974
403 433,1894,2063,2075,8077, 602,2741,8078,8079,8080,8081,8082,3014,1628,3400,8083, // 7990
404 3172,4574,4052,2890,4575,2512,8084,2544,2772,8085,8086,8087,3310,4576,2891,8088, // 8006
405 4577,8089,2851,4578,4579,1221,2967,4053,2513,8090,8091,8092,1867,1989,8093,8094, // 8022
406 8095,1895,8096,8097,4580,1896,4054, 318,8098,2094,4055,4293,8099,8100, 485,8101, // 8038
407 938,3862, 553,2670, 116,8102,3863,3612,8103,3498,2671,2773,3401,3311,2807,8104, // 8054
408 3613,2929,4056,1747,2930,2968,8105,8106, 207,8107,8108,2672,4581,2514,8109,3015, // 8070
409 890,3614,3864,8110,1877,3732,3402,8111,2183,2353,3403,1652,8112,8113,8114, 941, // 8086
410 2294, 208,3499,4057,2019, 330,4294,3865,2892,2492,3733,4295,8115,8116,8117,8118, // 8102
411
412 /***************************************************************************************
413 *Everything below is of no interest for detection purpose *
414 ***************************************************************************************
415
416 2515,1613,4582,8119,3312,3866,2516,8120,4058,8121,1637,4059,2466,4583,3867,8122, // 8118
417 2493,3016,3734,8123,8124,2192,8125,8126,2162,8127,8128,8129,8130,8131,8132,8133, // 8134
418 8134,8135,8136,8137,8138,8139,8140,8141,8142,8143,8144,8145,8146,8147,8148,8149, // 8150
419 8150,8151,8152,8153,8154,8155,8156,8157,8158,8159,8160,8161,8162,8163,8164,8165, // 8166
420 8166,8167,8168,8169,8170,8171,8172,8173,8174,8175,8176,8177,8178,8179,8180,8181, // 8182
421 8182,8183,8184,8185,8186,8187,8188,8189,8190,8191,8192,8193,8194,8195,8196,8197, // 8198
422 8198,8199,8200,8201,8202,8203,8204,8205,8206,8207,8208,8209,8210,8211,8212,8213, // 8214
423 8214,8215,8216,8217,8218,8219,8220,8221,8222,8223,8224,8225,8226,8227,8228,8229, // 8230
424 8230,8231,8232,8233,8234,8235,8236,8237,8238,8239,8240,8241,8242,8243,8244,8245, // 8246
425 8246,8247,8248,8249,8250,8251,8252,8253,8254,8255,8256,8257,8258,8259,8260,8261, // 8262
426 8262,8263,8264,8265,8266,8267,8268,8269,8270,8271,8272,8273,8274,8275,8276,8277, // 8278
427 8278,8279,8280,8281,8282,8283,8284,8285,8286,8287,8288,8289,8290,8291,8292,8293, // 8294
428 8294,8295,8296,8297,8298,8299,8300,8301,8302,8303,8304,8305,8306,8307,8308,8309, // 8310
429 8310,8311,8312,8313,8314,8315,8316,8317,8318,8319,8320,8321,8322,8323,8324,8325, // 8326
430 8326,8327,8328,8329,8330,8331,8332,8333,8334,8335,8336,8337,8338,8339,8340,8341, // 8342
431 8342,8343,8344,8345,8346,8347,8348,8349,8350,8351,8352,8353,8354,8355,8356,8357, // 8358
432 8358,8359,8360,8361,8362,8363,8364,8365,8366,8367,8368,8369,8370,8371,8372,8373, // 8374
433 8374,8375,8376,8377,8378,8379,8380,8381,8382,8383,8384,8385,8386,8387,8388,8389, // 8390
434 8390,8391,8392,8393,8394,8395,8396,8397,8398,8399,8400,8401,8402,8403,8404,8405, // 8406
435 8406,8407,8408,8409,8410,8411,8412,8413,8414,8415,8416,8417,8418,8419,8420,8421, // 8422
436 8422,8423,8424,8425,8426,8427,8428,8429,8430,8431,8432,8433,8434,8435,8436,8437, // 8438
437 8438,8439,8440,8441,8442,8443,8444,8445,8446,8447,8448,8449,8450,8451,8452,8453, // 8454
438 8454,8455,8456,8457,8458,8459,8460,8461,8462,8463,8464,8465,8466,8467,8468,8469, // 8470
439 8470,8471,8472,8473,8474,8475,8476,8477,8478,8479,8480,8481,8482,8483,8484,8485, // 8486
440 8486,8487,8488,8489,8490,8491,8492,8493,8494,8495,8496,8497,8498,8499,8500,8501, // 8502
441 8502,8503,8504,8505,8506,8507,8508,8509,8510,8511,8512,8513,8514,8515,8516,8517, // 8518
442 8518,8519,8520,8521,8522,8523,8524,8525,8526,8527,8528,8529,8530,8531,8532,8533, // 8534
443 8534,8535,8536,8537,8538,8539,8540,8541,8542,8543,8544,8545,8546,8547,8548,8549, // 8550
444 8550,8551,8552,8553,8554,8555,8556,8557,8558,8559,8560,8561,8562,8563,8564,8565, // 8566
445 8566,8567,8568,8569,8570,8571,8572,8573,8574,8575,8576,8577,8578,8579,8580,8581, // 8582
446 8582,8583,8584,8585,8586,8587,8588,8589,8590,8591,8592,8593,8594,8595,8596,8597, // 8598
447 8598,8599,8600,8601,8602,8603,8604,8605,8606,8607,8608,8609,8610,8611,8612,8613, // 8614
448 8614,8615,8616,8617,8618,8619,8620,8621,8622,8623,8624,8625,8626,8627,8628,8629, // 8630
449 8630,8631,8632,8633,8634,8635,8636,8637,8638,8639,8640,8641,8642,8643,8644,8645, // 8646
450 8646,8647,8648,8649,8650,8651,8652,8653,8654,8655,8656,8657,8658,8659,8660,8661, // 8662
451 8662,8663,8664,8665,8666,8667,8668,8669,8670,8671,8672,8673,8674,8675,8676,8677, // 8678
452 8678,8679,8680,8681,8682,8683,8684,8685,8686,8687,8688,8689,8690,8691,8692,8693, // 8694
453 8694,8695,8696,8697,8698,8699,8700,8701,8702,8703,8704,8705,8706,8707,8708,8709, // 8710
454 8710,8711,8712,8713,8714,8715,8716,8717,8718,8719,8720,8721,8722,8723,8724,8725, // 8726
455 8726,8727,8728,8729,8730,8731,8732,8733,8734,8735,8736,8737,8738,8739,8740,8741, // 8742
456 ****************************************************************************************/
457 };
458 }
+508
-0
src/main/java/org/mozilla/universalchardet/prober/distributionanalysis/GB2312DistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public class GB2312DistributionAnalysis extends CharDistributionAnalysis {
40 ////////////////////////////////////////////////////////////////
41 // constants
42 ////////////////////////////////////////////////////////////////
43 public static final float GB2312_TYPICAL_DISTRIBUTION_RATIO = 0.9f;
44 public static final int HIGHBYTE_BEGIN = 0xB0;
45 public static final int HIGHBYTE_END = 0xFE;
46 public static final int LOWBYTE_BEGIN = 0xA1;
47 public static final int LOWBYTE_END = 0xFE;
48
49
50 ////////////////////////////////////////////////////////////////
51 // methods
52 ////////////////////////////////////////////////////////////////
53 public GB2312DistributionAnalysis() {
54 super();
55 this.charToFreqOrder = gb2312CharToFreqOrder;
56 this.typicalDistributionRatio = GB2312_TYPICAL_DISTRIBUTION_RATIO;
57 }
58
59 @Override
60 protected int getOrder(byte[] buf, int offset) {
61 int highbyte = buf[offset] & 0xFF;
62 if (highbyte >= HIGHBYTE_BEGIN) {
63 int lowbyte = buf[offset+1] & 0xFF;
64 if (lowbyte >= LOWBYTE_BEGIN) {
65 return (94 * (highbyte - HIGHBYTE_BEGIN) + lowbyte - LOWBYTE_BEGIN);
66 }
67 }
68
69 return -1;
70 }
71
72
73 ////////////////////////////////////////////////////////////////
74 // constants continued
75 ////////////////////////////////////////////////////////////////
76 private static final int[] gb2312CharToFreqOrder = new int[] {
77 1671, 749,1443,2364,3924,3807,2330,3921,1704,3463,2691,1511,1515, 572,3191,2205,
78 2361, 224,2558, 479,1711, 963,3162, 440,4060,1905,2966,2947,3580,2647,3961,3842,
79 2204, 869,4207, 970,2678,5626,2944,2956,1479,4048, 514,3595, 588,1346,2820,3409,
80 249,4088,1746,1873,2047,1774, 581,1813, 358,1174,3590,1014,1561,4844,2245, 670,
81 1636,3112, 889,1286, 953, 556,2327,3060,1290,3141, 613, 185,3477,1367, 850,3820,
82 1715,2428,2642,2303,2732,3041,2562,2648,3566,3946,1349, 388,3098,2091,1360,3585,
83 152,1687,1539, 738,1559, 59,1232,2925,2267,1388,1249,1741,1679,2960, 151,1566,
84 1125,1352,4271, 924,4296, 385,3166,4459, 310,1245,2850, 70,3285,2729,3534,3575,
85 2398,3298,3466,1960,2265, 217,3647, 864,1909,2084,4401,2773,1010,3269,5152, 853,
86 3051,3121,1244,4251,1895, 364,1499,1540,2313,1180,3655,2268, 562, 715,2417,3061,
87 544, 336,3768,2380,1752,4075, 950, 280,2425,4382, 183,2759,3272, 333,4297,2155,
88 1688,2356,1444,1039,4540, 736,1177,3349,2443,2368,2144,2225, 565, 196,1482,3406,
89 927,1335,4147, 692, 878,1311,1653,3911,3622,1378,4200,1840,2969,3149,2126,1816,
90 2534,1546,2393,2760, 737,2494, 13, 447, 245,2747, 38,2765,2129,2589,1079, 606,
91 360, 471,3755,2890, 404, 848, 699,1785,1236, 370,2221,1023,3746,2074,2026,2023,
92 2388,1581,2119, 812,1141,3091,2536,1519, 804,2053, 406,1596,1090, 784, 548,4414,
93 1806,2264,2936,1100, 343,4114,5096, 622,3358, 743,3668,1510,1626,5020,3567,2513,
94 3195,4115,5627,2489,2991, 24,2065,2697,1087,2719, 48,1634, 315, 68, 985,2052,
95 198,2239,1347,1107,1439, 597,2366,2172, 871,3307, 919,2487,2790,1867, 236,2570,
96 1413,3794, 906,3365,3381,1701,1982,1818,1524,2924,1205, 616,2586,2072,2004, 575,
97 253,3099, 32,1365,1182, 197,1714,2454,1201, 554,3388,3224,2748, 756,2587, 250,
98 2567,1507,1517,3529,1922,2761,2337,3416,1961,1677,2452,2238,3153, 615, 911,1506,
99 1474,2495,1265,1906,2749,3756,3280,2161, 898,2714,1759,3450,2243,2444, 563, 26,
100 3286,2266,3769,3344,2707,3677, 611,1402, 531,1028,2871,4548,1375, 261,2948, 835,
101 1190,4134, 353, 840,2684,1900,3082,1435,2109,1207,1674, 329,1872,2781,4055,2686,
102 2104, 608,3318,2423,2957,2768,1108,3739,3512,3271,3985,2203,1771,3520,1418,2054,
103 1681,1153, 225,1627,2929, 162,2050,2511,3687,1954, 124,1859,2431,1684,3032,2894,
104 585,4805,3969,2869,2704,2088,2032,2095,3656,2635,4362,2209, 256, 518,2042,2105,
105 3777,3657, 643,2298,1148,1779, 190, 989,3544, 414, 11,2135,2063,2979,1471, 403,
106 3678, 126, 770,1563, 671,2499,3216,2877, 600,1179, 307,2805,4937,1268,1297,2694,
107 252,4032,1448,1494,1331,1394, 127,2256, 222,1647,1035,1481,3056,1915,1048, 873,
108 3651, 210, 33,1608,2516, 200,1520, 415, 102, 0,3389,1287, 817, 91,3299,2940,
109 836,1814, 549,2197,1396,1669,2987,3582,2297,2848,4528,1070, 687, 20,1819, 121,
110 1552,1364,1461,1968,2617,3540,2824,2083, 177, 948,4938,2291, 110,4549,2066, 648,
111 3359,1755,2110,2114,4642,4845,1693,3937,3308,1257,1869,2123, 208,1804,3159,2992,
112 2531,2549,3361,2418,1350,2347,2800,2568,1291,2036,2680, 72, 842,1990, 212,1233,
113 1154,1586, 75,2027,3410,4900,1823,1337,2710,2676, 728,2810,1522,3026,4995, 157,
114 755,1050,4022, 710, 785,1936,2194,2085,1406,2777,2400, 150,1250,4049,1206, 807,
115 1910, 534, 529,3309,1721,1660, 274, 39,2827, 661,2670,1578, 925,3248,3815,1094,
116 4278,4901,4252, 41,1150,3747,2572,2227,4501,3658,4902,3813,3357,3617,2884,2258,
117 887, 538,4187,3199,1294,2439,3042,2329,2343,2497,1255, 107, 543,1527, 521,3478,
118 3568, 194,5062, 15, 961,3870,1241,1192,2664, 66,5215,3260,2111,1295,1127,2152,
119 3805,4135, 901,1164,1976, 398,1278, 530,1460, 748, 904,1054,1966,1426, 53,2909,
120 509, 523,2279,1534, 536,1019, 239,1685, 460,2353, 673,1065,2401,3600,4298,2272,
121 1272,2363, 284,1753,3679,4064,1695, 81, 815,2677,2757,2731,1386, 859, 500,4221,
122 2190,2566, 757,1006,2519,2068,1166,1455, 337,2654,3203,1863,1682,1914,3025,1252,
123 1409,1366, 847, 714,2834,2038,3209, 964,2970,1901, 885,2553,1078,1756,3049, 301,
124 1572,3326, 688,2130,1996,2429,1805,1648,2930,3421,2750,3652,3088, 262,1158,1254,
125 389,1641,1812, 526,1719, 923,2073,1073,1902, 468, 489,4625,1140, 857,2375,3070,
126 3319,2863, 380, 116,1328,2693,1161,2244, 273,1212,1884,2769,3011,1775,1142, 461,
127 3066,1200,2147,2212, 790, 702,2695,4222,1601,1058, 434,2338,5153,3640, 67,2360,
128 4099,2502, 618,3472,1329, 416,1132, 830,2782,1807,2653,3211,3510,1662, 192,2124,
129 296,3979,1739,1611,3684, 23, 118, 324, 446,1239,1225, 293,2520,3814,3795,2535,
130 3116, 17,1074, 467,2692,2201, 387,2922, 45,1326,3055,1645,3659,2817, 958, 243,
131 1903,2320,1339,2825,1784,3289, 356, 576, 865,2315,2381,3377,3916,1088,3122,1713,
132 1655, 935, 628,4689,1034,1327, 441, 800, 720, 894,1979,2183,1528,5289,2702,1071,
133 4046,3572,2399,1571,3281, 79, 761,1103, 327, 134, 758,1899,1371,1615, 879, 442,
134 215,2605,2579, 173,2048,2485,1057,2975,3317,1097,2253,3801,4263,1403,1650,2946,
135 814,4968,3487,1548,2644,1567,1285, 2, 295,2636, 97, 946,3576, 832, 141,4257,
136 3273, 760,3821,3521,3156,2607, 949,1024,1733,1516,1803,1920,2125,2283,2665,3180,
137 1501,2064,3560,2171,1592, 803,3518,1416, 732,3897,4258,1363,1362,2458, 119,1427,
138 602,1525,2608,1605,1639,3175, 694,3064, 10, 465, 76,2000,4846,4208, 444,3781,
139 1619,3353,2206,1273,3796, 740,2483, 320,1723,2377,3660,2619,1359,1137,1762,1724,
140 2345,2842,1850,1862, 912, 821,1866, 612,2625,1735,2573,3369,1093, 844, 89, 937,
141 930,1424,3564,2413,2972,1004,3046,3019,2011, 711,3171,1452,4178, 428, 801,1943,
142 432, 445,2811, 206,4136,1472, 730, 349, 73, 397,2802,2547, 998,1637,1167, 789,
143 396,3217, 154,1218, 716,1120,1780,2819,4826,1931,3334,3762,2139,1215,2627, 552,
144 3664,3628,3232,1405,2383,3111,1356,2652,3577,3320,3101,1703, 640,1045,1370,1246,
145 4996, 371,1575,2436,1621,2210, 984,4033,1734,2638, 16,4529, 663,2755,3255,1451,
146 3917,2257,1253,1955,2234,1263,2951, 214,1229, 617, 485, 359,1831,1969, 473,2310,
147 750,2058, 165, 80,2864,2419, 361,4344,2416,2479,1134, 796,3726,1266,2943, 860,
148 2715, 938, 390,2734,1313,1384, 248, 202, 877,1064,2854, 522,3907, 279,1602, 297,
149 2357, 395,3740, 137,2075, 944,4089,2584,1267,3802, 62,1533,2285, 178, 176, 780,
150 2440, 201,3707, 590, 478,1560,4354,2117,1075, 30, 74,4643,4004,1635,1441,2745,
151 776,2596, 238,1077,1692,1912,2844, 605, 499,1742,3947, 241,3053, 980,1749, 936,
152 2640,4511,2582, 515,1543,2162,5322,2892,2993, 890,2148,1924, 665,1827,3581,1032,
153 968,3163, 339,1044,1896, 270, 583,1791,1720,4367,1194,3488,3669, 43,2523,1657,
154 163,2167, 290,1209,1622,3378, 550, 634,2508,2510, 695,2634,2384,2512,1476,1414,
155 220,1469,2341,2138,2852,3183,2900,4939,2865,3502,1211,3680, 854,3227,1299,2976,
156 3172, 186,2998,1459, 443,1067,3251,1495, 321,1932,3054, 909, 753,1410,1828, 436,
157 2441,1119,1587,3164,2186,1258, 227, 231,1425,1890,3200,3942, 247, 959, 725,5254,
158 2741, 577,2158,2079, 929, 120, 174, 838,2813, 591,1115, 417,2024, 40,3240,1536,
159 1037, 291,4151,2354, 632,1298,2406,2500,3535,1825,1846,3451, 205,1171, 345,4238,
160 18,1163, 811, 685,2208,1217, 425,1312,1508,1175,4308,2552,1033, 587,1381,3059,
161 2984,3482, 340,1316,4023,3972, 792,3176, 519, 777,4690, 918, 933,4130,2981,3741,
162 90,3360,2911,2200,5184,4550, 609,3079,2030, 272,3379,2736, 363,3881,1130,1447,
163 286, 779, 357,1169,3350,3137,1630,1220,2687,2391, 747,1277,3688,2618,2682,2601,
164 1156,3196,5290,4034,3102,1689,3596,3128, 874, 219,2783, 798, 508,1843,2461, 269,
165 1658,1776,1392,1913,2983,3287,2866,2159,2372, 829,4076, 46,4253,2873,1889,1894,
166 915,1834,1631,2181,2318, 298, 664,2818,3555,2735, 954,3228,3117, 527,3511,2173,
167 681,2712,3033,2247,2346,3467,1652, 155,2164,3382, 113,1994, 450, 899, 494, 994,
168 1237,2958,1875,2336,1926,3727, 545,1577,1550, 633,3473, 204,1305,3072,2410,1956,
169 2471, 707,2134, 841,2195,2196,2663,3843,1026,4940, 990,3252,4997, 368,1092, 437,
170 3212,3258,1933,1829, 675,2977,2893, 412, 943,3723,4644,3294,3283,2230,2373,5154,
171 2389,2241,2661,2323,1404,2524, 593, 787, 677,3008,1275,2059, 438,2709,2609,2240,
172 2269,2246,1446, 36,1568,1373,3892,1574,2301,1456,3962, 693,2276,5216,2035,1143,
173 2720,1919,1797,1811,2763,4137,2597,1830,1699,1488,1198,2090, 424,1694, 312,3634,
174 3390,4179,3335,2252,1214, 561,1059,3243,2295,2561, 975,5155,2321,2751,3772, 472,
175 1537,3282,3398,1047,2077,2348,2878,1323,3340,3076, 690,2906, 51, 369, 170,3541,
176 1060,2187,2688,3670,2541,1083,1683, 928,3918, 459, 109,4427, 599,3744,4286, 143,
177 2101,2730,2490, 82,1588,3036,2121, 281,1860, 477,4035,1238,2812,3020,2716,3312,
178 1530,2188,2055,1317, 843, 636,1808,1173,3495, 649, 181,1002, 147,3641,1159,2414,
179 3750,2289,2795, 813,3123,2610,1136,4368, 5,3391,4541,2174, 420, 429,1728, 754,
180 1228,2115,2219, 347,2223,2733, 735,1518,3003,2355,3134,1764,3948,3329,1888,2424,
181 1001,1234,1972,3321,3363,1672,1021,1450,1584, 226, 765, 655,2526,3404,3244,2302,
182 3665, 731, 594,2184, 319,1576, 621, 658,2656,4299,2099,3864,1279,2071,2598,2739,
183 795,3086,3699,3908,1707,2352,2402,1382,3136,2475,1465,4847,3496,3865,1085,3004,
184 2591,1084, 213,2287,1963,3565,2250, 822, 793,4574,3187,1772,1789,3050, 595,1484,
185 1959,2770,1080,2650, 456, 422,2996, 940,3322,4328,4345,3092,2742, 965,2784, 739,
186 4124, 952,1358,2498,2949,2565, 332,2698,2378, 660,2260,2473,4194,3856,2919, 535,
187 1260,2651,1208,1428,1300,1949,1303,2942, 433,2455,2450,1251,1946, 614,1269, 641,
188 1306,1810,2737,3078,2912, 564,2365,1419,1415,1497,4460,2367,2185,1379,3005,1307,
189 3218,2175,1897,3063, 682,1157,4040,4005,1712,1160,1941,1399, 394, 402,2952,1573,
190 1151,2986,2404, 862, 299,2033,1489,3006, 346, 171,2886,3401,1726,2932, 168,2533,
191 47,2507,1030,3735,1145,3370,1395,1318,1579,3609,4560,2857,4116,1457,2529,1965,
192 504,1036,2690,2988,2405, 745,5871, 849,2397,2056,3081, 863,2359,3857,2096, 99,
193 1397,1769,2300,4428,1643,3455,1978,1757,3718,1440, 35,4879,3742,1296,4228,2280,
194 160,5063,1599,2013, 166, 520,3479,1646,3345,3012, 490,1937,1545,1264,2182,2505,
195 1096,1188,1369,1436,2421,1667,2792,2460,1270,2122, 727,3167,2143, 806,1706,1012,
196 1800,3037, 960,2218,1882, 805, 139,2456,1139,1521, 851,1052,3093,3089, 342,2039,
197 744,5097,1468,1502,1585,2087, 223, 939, 326,2140,2577, 892,2481,1623,4077, 982,
198 3708, 135,2131, 87,2503,3114,2326,1106, 876,1616, 547,2997,2831,2093,3441,4530,
199 4314, 9,3256,4229,4148, 659,1462,1986,1710,2046,2913,2231,4090,4880,5255,3392,
200 3274,1368,3689,4645,1477, 705,3384,3635,1068,1529,2941,1458,3782,1509, 100,1656,
201 2548, 718,2339, 408,1590,2780,3548,1838,4117,3719,1345,3530, 717,3442,2778,3220,
202 2898,1892,4590,3614,3371,2043,1998,1224,3483, 891, 635, 584,2559,3355, 733,1766,
203 1729,1172,3789,1891,2307, 781,2982,2271,1957,1580,5773,2633,2005,4195,3097,1535,
204 3213,1189,1934,5693,3262, 586,3118,1324,1598, 517,1564,2217,1868,1893,4445,3728,
205 2703,3139,1526,1787,1992,3882,2875,1549,1199,1056,2224,1904,2711,5098,4287, 338,
206 1993,3129,3489,2689,1809,2815,1997, 957,1855,3898,2550,3275,3057,1105,1319, 627,
207 1505,1911,1883,3526, 698,3629,3456,1833,1431, 746, 77,1261,2017,2296,1977,1885,
208 125,1334,1600, 525,1798,1109,2222,1470,1945, 559,2236,1186,3443,2476,1929,1411,
209 2411,3135,1777,3372,2621,1841,1613,3229, 668,1430,1839,2643,2916, 195,1989,2671,
210 2358,1387, 629,3205,2293,5256,4439, 123,1310, 888,1879,4300,3021,3605,1003,1162,
211 3192,2910,2010, 140,2395,2859, 55,1082,2012,2901, 662, 419,2081,1438, 680,2774,
212 4654,3912,1620,1731,1625,5035,4065,2328, 512,1344, 802,5443,2163,2311,2537, 524,
213 3399, 98,1155,2103,1918,2606,3925,2816,1393,2465,1504,3773,2177,3963,1478,4346,
214 180,1113,4655,3461,2028,1698, 833,2696,1235,1322,1594,4408,3623,3013,3225,2040,
215 3022, 541,2881, 607,3632,2029,1665,1219, 639,1385,1686,1099,2803,3231,1938,3188,
216 2858, 427, 676,2772,1168,2025, 454,3253,2486,3556, 230,1950, 580, 791,1991,1280,
217 1086,1974,2034, 630, 257,3338,2788,4903,1017, 86,4790, 966,2789,1995,1696,1131,
218 259,3095,4188,1308, 179,1463,5257, 289,4107,1248, 42,3413,1725,2288, 896,1947,
219 774,4474,4254, 604,3430,4264, 392,2514,2588, 452, 237,1408,3018, 988,4531,1970,
220 3034,3310, 540,2370,1562,1288,2990, 502,4765,1147, 4,1853,2708, 207, 294,2814,
221 4078,2902,2509, 684, 34,3105,3532,2551, 644, 709,2801,2344, 573,1727,3573,3557,
222 2021,1081,3100,4315,2100,3681, 199,2263,1837,2385, 146,3484,1195,2776,3949, 997,
223 1939,3973,1008,1091,1202,1962,1847,1149,4209,5444,1076, 493, 117,5400,2521, 972,
224 1490,2934,1796,4542,2374,1512,2933,2657, 413,2888,1135,2762,2314,2156,1355,2369,
225 766,2007,2527,2170,3124,2491,2593,2632,4757,2437, 234,3125,3591,1898,1750,1376,
226 1942,3468,3138, 570,2127,2145,3276,4131, 962, 132,1445,4196, 19, 941,3624,3480,
227 3366,1973,1374,4461,3431,2629, 283,2415,2275, 808,2887,3620,2112,2563,1353,3610,
228 955,1089,3103,1053, 96, 88,4097, 823,3808,1583, 399, 292,4091,3313, 421,1128,
229 642,4006, 903,2539,1877,2082, 596, 29,4066,1790, 722,2157, 130, 995,1569, 769,
230 1485, 464, 513,2213, 288,1923,1101,2453,4316, 133, 486,2445, 50, 625, 487,2207,
231 57, 423, 481,2962, 159,3729,1558, 491, 303, 482, 501, 240,2837, 112,3648,2392,
232 1783, 362, 8,3433,3422, 610,2793,3277,1390,1284,1654, 21,3823, 734, 367, 623,
233 193, 287, 374,1009,1483, 816, 476, 313,2255,2340,1262,2150,2899,1146,2581, 782,
234 2116,1659,2018,1880, 255,3586,3314,1110,2867,2137,2564, 986,2767,5185,2006, 650,
235 158, 926, 762, 881,3157,2717,2362,3587, 306,3690,3245,1542,3077,2427,1691,2478,
236 2118,2985,3490,2438, 539,2305, 983, 129,1754, 355,4201,2386, 827,2923, 104,1773,
237 2838,2771, 411,2905,3919, 376, 767, 122,1114, 828,2422,1817,3506, 266,3460,1007,
238 1609,4998, 945,2612,4429,2274, 726,1247,1964,2914,2199,2070,4002,4108, 657,3323,
239 1422, 579, 455,2764,4737,1222,2895,1670, 824,1223,1487,2525, 558, 861,3080, 598,
240 2659,2515,1967, 752,2583,2376,2214,4180, 977, 704,2464,4999,2622,4109,1210,2961,
241 819,1541, 142,2284, 44, 418, 457,1126,3730,4347,4626,1644,1876,3671,1864, 302,
242 1063,5694, 624, 723,1984,3745,1314,1676,2488,1610,1449,3558,3569,2166,2098, 409,
243 1011,2325,3704,2306, 818,1732,1383,1824,1844,3757, 999,2705,3497,1216,1423,2683,
244 2426,2954,2501,2726,2229,1475,2554,5064,1971,1794,1666,2014,1343, 783, 724, 191,
245 2434,1354,2220,5065,1763,2752,2472,4152, 131, 175,2885,3434, 92,1466,4920,2616,
246 3871,3872,3866, 128,1551,1632, 669,1854,3682,4691,4125,1230, 188,2973,3290,1302,
247 1213, 560,3266, 917, 763,3909,3249,1760, 868,1958, 764,1782,2097, 145,2277,3774,
248 4462, 64,1491,3062, 971,2132,3606,2442, 221,1226,1617, 218, 323,1185,3207,3147,
249 571, 619,1473,1005,1744,2281, 449,1887,2396,3685, 275, 375,3816,1743,3844,3731,
250 845,1983,2350,4210,1377, 773, 967,3499,3052,3743,2725,4007,1697,1022,3943,1464,
251 3264,2855,2722,1952,1029,2839,2467, 84,4383,2215, 820,1391,2015,2448,3672, 377,
252 1948,2168, 797,2545,3536,2578,2645, 94,2874,1678, 405,1259,3071, 771, 546,1315,
253 470,1243,3083, 895,2468, 981, 969,2037, 846,4181, 653,1276,2928, 14,2594, 557,
254 3007,2474, 156, 902,1338,1740,2574, 537,2518, 973,2282,2216,2433,1928, 138,2903,
255 1293,2631,1612, 646,3457, 839,2935, 111, 496,2191,2847, 589,3186, 149,3994,2060,
256 4031,2641,4067,3145,1870, 37,3597,2136,1025,2051,3009,3383,3549,1121,1016,3261,
257 1301, 251,2446,2599,2153, 872,3246, 637, 334,3705, 831, 884, 921,3065,3140,4092,
258 2198,1944, 246,2964, 108,2045,1152,1921,2308,1031, 203,3173,4170,1907,3890, 810,
259 1401,2003,1690, 506, 647,1242,2828,1761,1649,3208,2249,1589,3709,2931,5156,1708,
260 498, 666,2613, 834,3817,1231, 184,2851,1124, 883,3197,2261,3710,1765,1553,2658,
261 1178,2639,2351, 93,1193, 942,2538,2141,4402, 235,1821, 870,1591,2192,1709,1871,
262 3341,1618,4126,2595,2334, 603, 651, 69, 701, 268,2662,3411,2555,1380,1606, 503,
263 448, 254,2371,2646, 574,1187,2309,1770, 322,2235,1292,1801, 305, 566,1133, 229,
264 2067,2057, 706, 167, 483,2002,2672,3295,1820,3561,3067, 316, 378,2746,3452,1112,
265 136,1981, 507,1651,2917,1117, 285,4591, 182,2580,3522,1304, 335,3303,1835,2504,
266 1795,1792,2248, 674,1018,2106,2449,1857,2292,2845, 976,3047,1781,2600,2727,1389,
267 1281, 52,3152, 153, 265,3950, 672,3485,3951,4463, 430,1183, 365, 278,2169, 27,
268 1407,1336,2304, 209,1340,1730,2202,1852,2403,2883, 979,1737,1062, 631,2829,2542,
269 3876,2592, 825,2086,2226,3048,3625, 352,1417,3724, 542, 991, 431,1351,3938,1861,
270 2294, 826,1361,2927,3142,3503,1738, 463,2462,2723, 582,1916,1595,2808, 400,3845,
271 3891,2868,3621,2254, 58,2492,1123, 910,2160,2614,1372,1603,1196,1072,3385,1700,
272 3267,1980, 696, 480,2430, 920, 799,1570,2920,1951,2041,4047,2540,1321,4223,2469,
273 3562,2228,1271,2602, 401,2833,3351,2575,5157, 907,2312,1256, 410, 263,3507,1582,
274 996, 678,1849,2316,1480, 908,3545,2237, 703,2322, 667,1826,2849,1531,2604,2999,
275 2407,3146,2151,2630,1786,3711, 469,3542, 497,3899,2409, 858, 837,4446,3393,1274,
276 786, 620,1845,2001,3311, 484, 308,3367,1204,1815,3691,2332,1532,2557,1842,2020,
277 2724,1927,2333,4440, 567, 22,1673,2728,4475,1987,1858,1144,1597, 101,1832,3601,
278 12, 974,3783,4391, 951,1412, 1,3720, 453,4608,4041, 528,1041,1027,3230,2628,
279 1129, 875,1051,3291,1203,2262,1069,2860,2799,2149,2615,3278, 144,1758,3040, 31,
280 475,1680, 366,2685,3184, 311,1642,4008,2466,5036,1593,1493,2809, 216,1420,1668,
281 233, 304,2128,3284, 232,1429,1768,1040,2008,3407,2740,2967,2543, 242,2133, 778,
282 1565,2022,2620, 505,2189,2756,1098,2273, 372,1614, 708, 553,2846,2094,2278, 169,
283 3626,2835,4161, 228,2674,3165, 809,1454,1309, 466,1705,1095, 900,3423, 880,2667,
284 3751,5258,2317,3109,2571,4317,2766,1503,1342, 866,4447,1118, 63,2076, 314,1881,
285 1348,1061, 172, 978,3515,1747, 532, 511,3970, 6, 601, 905,2699,3300,1751, 276,
286 1467,3725,2668, 65,4239,2544,2779,2556,1604, 578,2451,1802, 992,2331,2624,1320,
287 3446, 713,1513,1013, 103,2786,2447,1661, 886,1702, 916, 654,3574,2031,1556, 751,
288 2178,2821,2179,1498,1538,2176, 271, 914,2251,2080,1325, 638,1953,2937,3877,2432,
289 2754, 95,3265,1716, 260,1227,4083, 775, 106,1357,3254, 426,1607, 555,2480, 772,
290 1985, 244,2546, 474, 495,1046,2611,1851,2061, 71,2089,1675,2590, 742,3758,2843,
291 3222,1433, 267,2180,2576,2826,2233,2092,3913,2435, 956,1745,3075, 856,2113,1116,
292 451, 3,1988,2896,1398, 993,2463,1878,2049,1341,2718,2721,2870,2108, 712,2904,
293 4363,2753,2324, 277,2872,2349,2649, 384, 987, 435, 691,3000, 922, 164,3939, 652,
294 1500,1184,4153,2482,3373,2165,4848,2335,3775,3508,3154,2806,2830,1554,2102,1664,
295 2530,1434,2408, 893,1547,2623,3447,2832,2242,2532,3169,2856,3223,2078, 49,3770,
296 3469, 462, 318, 656,2259,3250,3069, 679,1629,2758, 344,1138,1104,3120,1836,1283,
297 3115,2154,1437,4448, 934, 759,1999, 794,2862,1038, 533,2560,1722,2342, 855,2626,
298 1197,1663,4476,3127, 85,4240,2528, 25,1111,1181,3673, 407,3470,4561,2679,2713,
299 768,1925,2841,3986,1544,1165, 932, 373,1240,2146,1930,2673, 721,4766, 354,4333,
300 391,2963, 187, 61,3364,1442,1102, 330,1940,1767, 341,3809,4118, 393,2496,2062,
301 2211, 105, 331, 300, 439, 913,1332, 626, 379,3304,1557, 328, 689,3952, 309,1555,
302 931, 317,2517,3027, 325, 569, 686,2107,3084, 60,1042,1333,2794, 264,3177,4014,
303 1628, 258,3712, 7,4464,1176,1043,1778, 683, 114,1975, 78,1492, 383,1886, 510,
304 386, 645,5291,2891,2069,3305,4138,3867,2939,2603,2493,1935,1066,1848,3588,1015,
305 1282,1289,4609, 697,1453,3044,2666,3611,1856,2412, 54, 719,1330, 568,3778,2459,
306 1748, 788, 492, 551,1191,1000, 488,3394,3763, 282,1799, 348,2016,1523,3155,2390,
307 1049, 382,2019,1788,1170, 729,2968,3523, 897,3926,2785,2938,3292, 350,2319,3238,
308 1718,1717,2655,3453,3143,4465, 161,2889,2980,2009,1421, 56,1908,1640,2387,2232,
309 1917,1874,2477,4921, 148, 83,3438, 592,4245,2882,1822,1055, 741, 115,1496,1624,
310 381,1638,4592,1020, 516,3214, 458, 947,4575,1432, 211,1514,2926,1865,2142, 189,
311 852,1221,1400,1486, 882,2299,4036, 351, 28,1122, 700,6479,6480,6481,6482,6483, //last 512
312
313 /***************************************************************************************
314 *Everything below is of no interest for detection purpose *
315 ***************************************************************************************
316
317 5508,6484,3900,3414,3974,4441,4024,3537,4037,5628,5099,3633,6485,3148,6486,3636,
318 5509,3257,5510,5973,5445,5872,4941,4403,3174,4627,5873,6276,2286,4230,5446,5874,
319 5122,6102,6103,4162,5447,5123,5323,4849,6277,3980,3851,5066,4246,5774,5067,6278,
320 3001,2807,5695,3346,5775,5974,5158,5448,6487,5975,5976,5776,3598,6279,5696,4806,
321 4211,4154,6280,6488,6489,6490,6281,4212,5037,3374,4171,6491,4562,4807,4722,4827,
322 5977,6104,4532,4079,5159,5324,5160,4404,3858,5359,5875,3975,4288,4610,3486,4512,
323 5325,3893,5360,6282,6283,5560,2522,4231,5978,5186,5449,2569,3878,6284,5401,3578,
324 4415,6285,4656,5124,5979,2506,4247,4449,3219,3417,4334,4969,4329,6492,4576,4828,
325 4172,4416,4829,5402,6286,3927,3852,5361,4369,4830,4477,4867,5876,4173,6493,6105,
326 4657,6287,6106,5877,5450,6494,4155,4868,5451,3700,5629,4384,6288,6289,5878,3189,
327 4881,6107,6290,6495,4513,6496,4692,4515,4723,5100,3356,6497,6291,3810,4080,5561,
328 3570,4430,5980,6498,4355,5697,6499,4724,6108,6109,3764,4050,5038,5879,4093,3226,
329 6292,5068,5217,4693,3342,5630,3504,4831,4377,4466,4309,5698,4431,5777,6293,5778,
330 4272,3706,6110,5326,3752,4676,5327,4273,5403,4767,5631,6500,5699,5880,3475,5039,
331 6294,5562,5125,4348,4301,4482,4068,5126,4593,5700,3380,3462,5981,5563,3824,5404,
332 4970,5511,3825,4738,6295,6501,5452,4516,6111,5881,5564,6502,6296,5982,6503,4213,
333 4163,3454,6504,6112,4009,4450,6113,4658,6297,6114,3035,6505,6115,3995,4904,4739,
334 4563,4942,4110,5040,3661,3928,5362,3674,6506,5292,3612,4791,5565,4149,5983,5328,
335 5259,5021,4725,4577,4564,4517,4364,6298,5405,4578,5260,4594,4156,4157,5453,3592,
336 3491,6507,5127,5512,4709,4922,5984,5701,4726,4289,6508,4015,6116,5128,4628,3424,
337 4241,5779,6299,4905,6509,6510,5454,5702,5780,6300,4365,4923,3971,6511,5161,3270,
338 3158,5985,4100, 867,5129,5703,6117,5363,3695,3301,5513,4467,6118,6512,5455,4232,
339 4242,4629,6513,3959,4478,6514,5514,5329,5986,4850,5162,5566,3846,4694,6119,5456,
340 4869,5781,3779,6301,5704,5987,5515,4710,6302,5882,6120,4392,5364,5705,6515,6121,
341 6516,6517,3736,5988,5457,5989,4695,2457,5883,4551,5782,6303,6304,6305,5130,4971,
342 6122,5163,6123,4870,3263,5365,3150,4871,6518,6306,5783,5069,5706,3513,3498,4409,
343 5330,5632,5366,5458,5459,3991,5990,4502,3324,5991,5784,3696,4518,5633,4119,6519,
344 4630,5634,4417,5707,4832,5992,3418,6124,5993,5567,4768,5218,6520,4595,3458,5367,
345 6125,5635,6126,4202,6521,4740,4924,6307,3981,4069,4385,6308,3883,2675,4051,3834,
346 4302,4483,5568,5994,4972,4101,5368,6309,5164,5884,3922,6127,6522,6523,5261,5460,
347 5187,4164,5219,3538,5516,4111,3524,5995,6310,6311,5369,3181,3386,2484,5188,3464,
348 5569,3627,5708,6524,5406,5165,4677,4492,6312,4872,4851,5885,4468,5996,6313,5709,
349 5710,6128,2470,5886,6314,5293,4882,5785,3325,5461,5101,6129,5711,5786,6525,4906,
350 6526,6527,4418,5887,5712,4808,2907,3701,5713,5888,6528,3765,5636,5331,6529,6530,
351 3593,5889,3637,4943,3692,5714,5787,4925,6315,6130,5462,4405,6131,6132,6316,5262,
352 6531,6532,5715,3859,5716,5070,4696,5102,3929,5788,3987,4792,5997,6533,6534,3920,
353 4809,5000,5998,6535,2974,5370,6317,5189,5263,5717,3826,6536,3953,5001,4883,3190,
354 5463,5890,4973,5999,4741,6133,6134,3607,5570,6000,4711,3362,3630,4552,5041,6318,
355 6001,2950,2953,5637,4646,5371,4944,6002,2044,4120,3429,6319,6537,5103,4833,6538,
356 6539,4884,4647,3884,6003,6004,4758,3835,5220,5789,4565,5407,6540,6135,5294,4697,
357 4852,6320,6321,3206,4907,6541,6322,4945,6542,6136,6543,6323,6005,4631,3519,6544,
358 5891,6545,5464,3784,5221,6546,5571,4659,6547,6324,6137,5190,6548,3853,6549,4016,
359 4834,3954,6138,5332,3827,4017,3210,3546,4469,5408,5718,3505,4648,5790,5131,5638,
360 5791,5465,4727,4318,6325,6326,5792,4553,4010,4698,3439,4974,3638,4335,3085,6006,
361 5104,5042,5166,5892,5572,6327,4356,4519,5222,5573,5333,5793,5043,6550,5639,5071,
362 4503,6328,6139,6551,6140,3914,3901,5372,6007,5640,4728,4793,3976,3836,4885,6552,
363 4127,6553,4451,4102,5002,6554,3686,5105,6555,5191,5072,5295,4611,5794,5296,6556,
364 5893,5264,5894,4975,5466,5265,4699,4976,4370,4056,3492,5044,4886,6557,5795,4432,
365 4769,4357,5467,3940,4660,4290,6141,4484,4770,4661,3992,6329,4025,4662,5022,4632,
366 4835,4070,5297,4663,4596,5574,5132,5409,5895,6142,4504,5192,4664,5796,5896,3885,
367 5575,5797,5023,4810,5798,3732,5223,4712,5298,4084,5334,5468,6143,4052,4053,4336,
368 4977,4794,6558,5335,4908,5576,5224,4233,5024,4128,5469,5225,4873,6008,5045,4729,
369 4742,4633,3675,4597,6559,5897,5133,5577,5003,5641,5719,6330,6560,3017,2382,3854,
370 4406,4811,6331,4393,3964,4946,6561,2420,3722,6562,4926,4378,3247,1736,4442,6332,
371 5134,6333,5226,3996,2918,5470,4319,4003,4598,4743,4744,4485,3785,3902,5167,5004,
372 5373,4394,5898,6144,4874,1793,3997,6334,4085,4214,5106,5642,4909,5799,6009,4419,
373 4189,3330,5899,4165,4420,5299,5720,5227,3347,6145,4081,6335,2876,3930,6146,3293,
374 3786,3910,3998,5900,5300,5578,2840,6563,5901,5579,6147,3531,5374,6564,6565,5580,
375 4759,5375,6566,6148,3559,5643,6336,6010,5517,6337,6338,5721,5902,3873,6011,6339,
376 6567,5518,3868,3649,5722,6568,4771,4947,6569,6149,4812,6570,2853,5471,6340,6341,
377 5644,4795,6342,6012,5723,6343,5724,6013,4349,6344,3160,6150,5193,4599,4514,4493,
378 5168,4320,6345,4927,3666,4745,5169,5903,5005,4928,6346,5725,6014,4730,4203,5046,
379 4948,3395,5170,6015,4150,6016,5726,5519,6347,5047,3550,6151,6348,4197,4310,5904,
380 6571,5581,2965,6152,4978,3960,4291,5135,6572,5301,5727,4129,4026,5905,4853,5728,
381 5472,6153,6349,4533,2700,4505,5336,4678,3583,5073,2994,4486,3043,4554,5520,6350,
382 6017,5800,4487,6351,3931,4103,5376,6352,4011,4321,4311,4190,5136,6018,3988,3233,
383 4350,5906,5645,4198,6573,5107,3432,4191,3435,5582,6574,4139,5410,6353,5411,3944,
384 5583,5074,3198,6575,6354,4358,6576,5302,4600,5584,5194,5412,6577,6578,5585,5413,
385 5303,4248,5414,3879,4433,6579,4479,5025,4854,5415,6355,4760,4772,3683,2978,4700,
386 3797,4452,3965,3932,3721,4910,5801,6580,5195,3551,5907,3221,3471,3029,6019,3999,
387 5908,5909,5266,5267,3444,3023,3828,3170,4796,5646,4979,4259,6356,5647,5337,3694,
388 6357,5648,5338,4520,4322,5802,3031,3759,4071,6020,5586,4836,4386,5048,6581,3571,
389 4679,4174,4949,6154,4813,3787,3402,3822,3958,3215,3552,5268,4387,3933,4950,4359,
390 6021,5910,5075,3579,6358,4234,4566,5521,6359,3613,5049,6022,5911,3375,3702,3178,
391 4911,5339,4521,6582,6583,4395,3087,3811,5377,6023,6360,6155,4027,5171,5649,4421,
392 4249,2804,6584,2270,6585,4000,4235,3045,6156,5137,5729,4140,4312,3886,6361,4330,
393 6157,4215,6158,3500,3676,4929,4331,3713,4930,5912,4265,3776,3368,5587,4470,4855,
394 3038,4980,3631,6159,6160,4132,4680,6161,6362,3923,4379,5588,4255,6586,4121,6587,
395 6363,4649,6364,3288,4773,4774,6162,6024,6365,3543,6588,4274,3107,3737,5050,5803,
396 4797,4522,5589,5051,5730,3714,4887,5378,4001,4523,6163,5026,5522,4701,4175,2791,
397 3760,6589,5473,4224,4133,3847,4814,4815,4775,3259,5416,6590,2738,6164,6025,5304,
398 3733,5076,5650,4816,5590,6591,6165,6592,3934,5269,6593,3396,5340,6594,5804,3445,
399 3602,4042,4488,5731,5732,3525,5591,4601,5196,6166,6026,5172,3642,4612,3202,4506,
400 4798,6366,3818,5108,4303,5138,5139,4776,3332,4304,2915,3415,4434,5077,5109,4856,
401 2879,5305,4817,6595,5913,3104,3144,3903,4634,5341,3133,5110,5651,5805,6167,4057,
402 5592,2945,4371,5593,6596,3474,4182,6367,6597,6168,4507,4279,6598,2822,6599,4777,
403 4713,5594,3829,6169,3887,5417,6170,3653,5474,6368,4216,2971,5228,3790,4579,6369,
404 5733,6600,6601,4951,4746,4555,6602,5418,5475,6027,3400,4665,5806,6171,4799,6028,
405 5052,6172,3343,4800,4747,5006,6370,4556,4217,5476,4396,5229,5379,5477,3839,5914,
406 5652,5807,4714,3068,4635,5808,6173,5342,4192,5078,5419,5523,5734,6174,4557,6175,
407 4602,6371,6176,6603,5809,6372,5735,4260,3869,5111,5230,6029,5112,6177,3126,4681,
408 5524,5915,2706,3563,4748,3130,6178,4018,5525,6604,6605,5478,4012,4837,6606,4534,
409 4193,5810,4857,3615,5479,6030,4082,3697,3539,4086,5270,3662,4508,4931,5916,4912,
410 5811,5027,3888,6607,4397,3527,3302,3798,2775,2921,2637,3966,4122,4388,4028,4054,
411 1633,4858,5079,3024,5007,3982,3412,5736,6608,3426,3236,5595,3030,6179,3427,3336,
412 3279,3110,6373,3874,3039,5080,5917,5140,4489,3119,6374,5812,3405,4494,6031,4666,
413 4141,6180,4166,6032,5813,4981,6609,5081,4422,4982,4112,3915,5653,3296,3983,6375,
414 4266,4410,5654,6610,6181,3436,5082,6611,5380,6033,3819,5596,4535,5231,5306,5113,
415 6612,4952,5918,4275,3113,6613,6376,6182,6183,5814,3073,4731,4838,5008,3831,6614,
416 4888,3090,3848,4280,5526,5232,3014,5655,5009,5737,5420,5527,6615,5815,5343,5173,
417 5381,4818,6616,3151,4953,6617,5738,2796,3204,4360,2989,4281,5739,5174,5421,5197,
418 3132,5141,3849,5142,5528,5083,3799,3904,4839,5480,2880,4495,3448,6377,6184,5271,
419 5919,3771,3193,6034,6035,5920,5010,6036,5597,6037,6378,6038,3106,5422,6618,5423,
420 5424,4142,6619,4889,5084,4890,4313,5740,6620,3437,5175,5307,5816,4199,5198,5529,
421 5817,5199,5656,4913,5028,5344,3850,6185,2955,5272,5011,5818,4567,4580,5029,5921,
422 3616,5233,6621,6622,6186,4176,6039,6379,6380,3352,5200,5273,2908,5598,5234,3837,
423 5308,6623,6624,5819,4496,4323,5309,5201,6625,6626,4983,3194,3838,4167,5530,5922,
424 5274,6381,6382,3860,3861,5599,3333,4292,4509,6383,3553,5481,5820,5531,4778,6187,
425 3955,3956,4324,4389,4218,3945,4325,3397,2681,5923,4779,5085,4019,5482,4891,5382,
426 5383,6040,4682,3425,5275,4094,6627,5310,3015,5483,5657,4398,5924,3168,4819,6628,
427 5925,6629,5532,4932,4613,6041,6630,4636,6384,4780,4204,5658,4423,5821,3989,4683,
428 5822,6385,4954,6631,5345,6188,5425,5012,5384,3894,6386,4490,4104,6632,5741,5053,
429 6633,5823,5926,5659,5660,5927,6634,5235,5742,5824,4840,4933,4820,6387,4859,5928,
430 4955,6388,4143,3584,5825,5346,5013,6635,5661,6389,5014,5484,5743,4337,5176,5662,
431 6390,2836,6391,3268,6392,6636,6042,5236,6637,4158,6638,5744,5663,4471,5347,3663,
432 4123,5143,4293,3895,6639,6640,5311,5929,5826,3800,6189,6393,6190,5664,5348,3554,
433 3594,4749,4603,6641,5385,4801,6043,5827,4183,6642,5312,5426,4761,6394,5665,6191,
434 4715,2669,6643,6644,5533,3185,5427,5086,5930,5931,5386,6192,6044,6645,4781,4013,
435 5745,4282,4435,5534,4390,4267,6045,5746,4984,6046,2743,6193,3501,4087,5485,5932,
436 5428,4184,4095,5747,4061,5054,3058,3862,5933,5600,6646,5144,3618,6395,3131,5055,
437 5313,6396,4650,4956,3855,6194,3896,5202,4985,4029,4225,6195,6647,5828,5486,5829,
438 3589,3002,6648,6397,4782,5276,6649,6196,6650,4105,3803,4043,5237,5830,6398,4096,
439 3643,6399,3528,6651,4453,3315,4637,6652,3984,6197,5535,3182,3339,6653,3096,2660,
440 6400,6654,3449,5934,4250,4236,6047,6401,5831,6655,5487,3753,4062,5832,6198,6199,
441 6656,3766,6657,3403,4667,6048,6658,4338,2897,5833,3880,2797,3780,4326,6659,5748,
442 5015,6660,5387,4351,5601,4411,6661,3654,4424,5935,4339,4072,5277,4568,5536,6402,
443 6662,5238,6663,5349,5203,6200,5204,6201,5145,4536,5016,5056,4762,5834,4399,4957,
444 6202,6403,5666,5749,6664,4340,6665,5936,5177,5667,6666,6667,3459,4668,6404,6668,
445 6669,4543,6203,6670,4276,6405,4480,5537,6671,4614,5205,5668,6672,3348,2193,4763,
446 6406,6204,5937,5602,4177,5669,3419,6673,4020,6205,4443,4569,5388,3715,3639,6407,
447 6049,4058,6206,6674,5938,4544,6050,4185,4294,4841,4651,4615,5488,6207,6408,6051,
448 5178,3241,3509,5835,6208,4958,5836,4341,5489,5278,6209,2823,5538,5350,5206,5429,
449 6675,4638,4875,4073,3516,4684,4914,4860,5939,5603,5389,6052,5057,3237,5490,3791,
450 6676,6409,6677,4821,4915,4106,5351,5058,4243,5539,4244,5604,4842,4916,5239,3028,
451 3716,5837,5114,5605,5390,5940,5430,6210,4332,6678,5540,4732,3667,3840,6053,4305,
452 3408,5670,5541,6410,2744,5240,5750,6679,3234,5606,6680,5607,5671,3608,4283,4159,
453 4400,5352,4783,6681,6411,6682,4491,4802,6211,6412,5941,6413,6414,5542,5751,6683,
454 4669,3734,5942,6684,6415,5943,5059,3328,4670,4144,4268,6685,6686,6687,6688,4372,
455 3603,6689,5944,5491,4373,3440,6416,5543,4784,4822,5608,3792,4616,5838,5672,3514,
456 5391,6417,4892,6690,4639,6691,6054,5673,5839,6055,6692,6056,5392,6212,4038,5544,
457 5674,4497,6057,6693,5840,4284,5675,4021,4545,5609,6418,4454,6419,6213,4113,4472,
458 5314,3738,5087,5279,4074,5610,4959,4063,3179,4750,6058,6420,6214,3476,4498,4716,
459 5431,4960,4685,6215,5241,6694,6421,6216,6695,5841,5945,6422,3748,5946,5179,3905,
460 5752,5545,5947,4374,6217,4455,6423,4412,6218,4803,5353,6696,3832,5280,6219,4327,
461 4702,6220,6221,6059,4652,5432,6424,3749,4751,6425,5753,4986,5393,4917,5948,5030,
462 5754,4861,4733,6426,4703,6697,6222,4671,5949,4546,4961,5180,6223,5031,3316,5281,
463 6698,4862,4295,4934,5207,3644,6427,5842,5950,6428,6429,4570,5843,5282,6430,6224,
464 5088,3239,6060,6699,5844,5755,6061,6431,2701,5546,6432,5115,5676,4039,3993,3327,
465 4752,4425,5315,6433,3941,6434,5677,4617,4604,3074,4581,6225,5433,6435,6226,6062,
466 4823,5756,5116,6227,3717,5678,4717,5845,6436,5679,5846,6063,5847,6064,3977,3354,
467 6437,3863,5117,6228,5547,5394,4499,4524,6229,4605,6230,4306,4500,6700,5951,6065,
468 3693,5952,5089,4366,4918,6701,6231,5548,6232,6702,6438,4704,5434,6703,6704,5953,
469 4168,6705,5680,3420,6706,5242,4407,6066,3812,5757,5090,5954,4672,4525,3481,5681,
470 4618,5395,5354,5316,5955,6439,4962,6707,4526,6440,3465,4673,6067,6441,5682,6708,
471 5435,5492,5758,5683,4619,4571,4674,4804,4893,4686,5493,4753,6233,6068,4269,6442,
472 6234,5032,4705,5146,5243,5208,5848,6235,6443,4963,5033,4640,4226,6236,5849,3387,
473 6444,6445,4436,4437,5850,4843,5494,4785,4894,6709,4361,6710,5091,5956,3331,6237,
474 4987,5549,6069,6711,4342,3517,4473,5317,6070,6712,6071,4706,6446,5017,5355,6713,
475 6714,4988,5436,6447,4734,5759,6715,4735,4547,4456,4754,6448,5851,6449,6450,3547,
476 5852,5318,6451,6452,5092,4205,6716,6238,4620,4219,5611,6239,6072,4481,5760,5957,
477 5958,4059,6240,6453,4227,4537,6241,5761,4030,4186,5244,5209,3761,4457,4876,3337,
478 5495,5181,6242,5959,5319,5612,5684,5853,3493,5854,6073,4169,5613,5147,4895,6074,
479 5210,6717,5182,6718,3830,6243,2798,3841,6075,6244,5855,5614,3604,4606,5496,5685,
480 5118,5356,6719,6454,5960,5357,5961,6720,4145,3935,4621,5119,5962,4261,6721,6455,
481 4786,5963,4375,4582,6245,6246,6247,6076,5437,4877,5856,3376,4380,6248,4160,6722,
482 5148,6456,5211,6457,6723,4718,6458,6724,6249,5358,4044,3297,6459,6250,5857,5615,
483 5497,5245,6460,5498,6725,6251,6252,5550,3793,5499,2959,5396,6461,6462,4572,5093,
484 5500,5964,3806,4146,6463,4426,5762,5858,6077,6253,4755,3967,4220,5965,6254,4989,
485 5501,6464,4352,6726,6078,4764,2290,5246,3906,5438,5283,3767,4964,2861,5763,5094,
486 6255,6256,4622,5616,5859,5860,4707,6727,4285,4708,4824,5617,6257,5551,4787,5212,
487 4965,4935,4687,6465,6728,6466,5686,6079,3494,4413,2995,5247,5966,5618,6729,5967,
488 5764,5765,5687,5502,6730,6731,6080,5397,6467,4990,6258,6732,4538,5060,5619,6733,
489 4719,5688,5439,5018,5149,5284,5503,6734,6081,4607,6259,5120,3645,5861,4583,6260,
490 4584,4675,5620,4098,5440,6261,4863,2379,3306,4585,5552,5689,4586,5285,6735,4864,
491 6736,5286,6082,6737,4623,3010,4788,4381,4558,5621,4587,4896,3698,3161,5248,4353,
492 4045,6262,3754,5183,4588,6738,6263,6739,6740,5622,3936,6741,6468,6742,6264,5095,
493 6469,4991,5968,6743,4992,6744,6083,4897,6745,4256,5766,4307,3108,3968,4444,5287,
494 3889,4343,6084,4510,6085,4559,6086,4898,5969,6746,5623,5061,4919,5249,5250,5504,
495 5441,6265,5320,4878,3242,5862,5251,3428,6087,6747,4237,5624,5442,6266,5553,4539,
496 6748,2585,3533,5398,4262,6088,5150,4736,4438,6089,6267,5505,4966,6749,6268,6750,
497 6269,5288,5554,3650,6090,6091,4624,6092,5690,6751,5863,4270,5691,4277,5555,5864,
498 6752,5692,4720,4865,6470,5151,4688,4825,6753,3094,6754,6471,3235,4653,6755,5213,
499 5399,6756,3201,4589,5865,4967,6472,5866,6473,5019,3016,6757,5321,4756,3957,4573,
500 6093,4993,5767,4721,6474,6758,5625,6759,4458,6475,6270,6760,5556,4994,5214,5252,
501 6271,3875,5768,6094,5034,5506,4376,5769,6761,2120,6476,5253,5770,6762,5771,5970,
502 3990,5971,5557,5558,5772,6477,6095,2787,4641,5972,5121,6096,6097,6272,6763,3703,
503 5867,5507,6273,4206,6274,4789,6098,6764,3619,3646,3833,3804,2394,3788,4936,3978,
504 4866,4899,6099,6100,5559,6478,6765,3599,5868,6101,5869,5870,6275,6766,4527,6767,
505 *******************************************************************************/
506 };
507 }
+585
-0
src/main/java/org/mozilla/universalchardet/prober/distributionanalysis/JISDistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public abstract class JISDistributionAnalysis extends CharDistributionAnalysis {
40 ////////////////////////////////////////////////////////////////
41 // constants
42 ////////////////////////////////////////////////////////////////
43 public static final float JIS_TYPICAL_DISTRIBUTION_RATIO = 3.0f;
44
45
46 ////////////////////////////////////////////////////////////////
47 // methods
48 ////////////////////////////////////////////////////////////////
49 public JISDistributionAnalysis() {
50 super();
51 this.typicalDistributionRatio = JIS_TYPICAL_DISTRIBUTION_RATIO;
52 this.charToFreqOrder = jisCharToFreqOrder;
53 }
54
55
56 ////////////////////////////////////////////////////////////////
57 // constants continued
58 ////////////////////////////////////////////////////////////////
59 private static final int[] jisCharToFreqOrder = new int[] {
60 40, 1, 6, 182, 152, 180, 295,2127, 285, 381,3295,4304,3068,4606,3165,3510, // 16
61 3511,1822,2785,4607,1193,2226,5070,4608, 171,2996,1247, 18, 179,5071, 856,1661, // 32
62 1262,5072, 619, 127,3431,3512,3230,1899,1700, 232, 228,1294,1298, 284, 283,2041, // 48
63 2042,1061,1062, 48, 49, 44, 45, 433, 434,1040,1041, 996, 787,2997,1255,4305, // 64
64 2108,4609,1684,1648,5073,5074,5075,5076,5077,5078,3687,5079,4610,5080,3927,3928, // 80
65 5081,3296,3432, 290,2285,1471,2187,5082,2580,2825,1303,2140,1739,1445,2691,3375, // 96
66 1691,3297,4306,4307,4611, 452,3376,1182,2713,3688,3069,4308,5083,5084,5085,5086, // 112
67 5087,5088,5089,5090,5091,5092,5093,5094,5095,5096,5097,5098,5099,5100,5101,5102, // 128
68 5103,5104,5105,5106,5107,5108,5109,5110,5111,5112,4097,5113,5114,5115,5116,5117, // 144
69 5118,5119,5120,5121,5122,5123,5124,5125,5126,5127,5128,5129,5130,5131,5132,5133, // 160
70 5134,5135,5136,5137,5138,5139,5140,5141,5142,5143,5144,5145,5146,5147,5148,5149, // 176
71 5150,5151,5152,4612,5153,5154,5155,5156,5157,5158,5159,5160,5161,5162,5163,5164, // 192
72 5165,5166,5167,5168,5169,5170,5171,5172,5173,5174,5175,1472, 598, 618, 820,1205, // 208
73 1309,1412,1858,1307,1692,5176,5177,5178,5179,5180,5181,5182,1142,1452,1234,1172, // 224
74 1875,2043,2149,1793,1382,2973, 925,2404,1067,1241, 960,1377,2935,1491, 919,1217, // 240
75 1865,2030,1406,1499,2749,4098,5183,5184,5185,5186,5187,5188,2561,4099,3117,1804, // 256
76 2049,3689,4309,3513,1663,5189,3166,3118,3298,1587,1561,3433,5190,3119,1625,2998, // 272
77 3299,4613,1766,3690,2786,4614,5191,5192,5193,5194,2161, 26,3377, 2,3929, 20, // 288
78 3691, 47,4100, 50, 17, 16, 35, 268, 27, 243, 42, 155, 24, 154, 29, 184, // 304
79 4, 91, 14, 92, 53, 396, 33, 289, 9, 37, 64, 620, 21, 39, 321, 5, // 320
80 12, 11, 52, 13, 3, 208, 138, 0, 7, 60, 526, 141, 151,1069, 181, 275, // 336
81 1591, 83, 132,1475, 126, 331, 829, 15, 69, 160, 59, 22, 157, 55,1079, 312, // 352
82 109, 38, 23, 25, 10, 19, 79,5195, 61, 382,1124, 8, 30,5196,5197,5198, // 368
83 5199,5200,5201,5202,5203,5204,5205,5206, 89, 62, 74, 34,2416, 112, 139, 196, // 384
84 271, 149, 84, 607, 131, 765, 46, 88, 153, 683, 76, 874, 101, 258, 57, 80, // 400
85 32, 364, 121,1508, 169,1547, 68, 235, 145,2999, 41, 360,3027, 70, 63, 31, // 416
86 43, 259, 262,1383, 99, 533, 194, 66, 93, 846, 217, 192, 56, 106, 58, 565, // 432
87 280, 272, 311, 256, 146, 82, 308, 71, 100, 128, 214, 655, 110, 261, 104,1140, // 448
88 54, 51, 36, 87, 67,3070, 185,2618,2936,2020, 28,1066,2390,2059,5207,5208, // 464
89 5209,5210,5211,5212,5213,5214,5215,5216,4615,5217,5218,5219,5220,5221,5222,5223, // 480
90 5224,5225,5226,5227,5228,5229,5230,5231,5232,5233,5234,5235,5236,3514,5237,5238, // 496
91 5239,5240,5241,5242,5243,5244,2297,2031,4616,4310,3692,5245,3071,5246,3598,5247, // 512
92 4617,3231,3515,5248,4101,4311,4618,3808,4312,4102,5249,4103,4104,3599,5250,5251, // 528
93 5252,5253,5254,5255,5256,5257,5258,5259,5260,5261,5262,5263,5264,5265,5266,5267, // 544
94 5268,5269,5270,5271,5272,5273,5274,5275,5276,5277,5278,5279,5280,5281,5282,5283, // 560
95 5284,5285,5286,5287,5288,5289,5290,5291,5292,5293,5294,5295,5296,5297,5298,5299, // 576
96 5300,5301,5302,5303,5304,5305,5306,5307,5308,5309,5310,5311,5312,5313,5314,5315, // 592
97 5316,5317,5318,5319,5320,5321,5322,5323,5324,5325,5326,5327,5328,5329,5330,5331, // 608
98 5332,5333,5334,5335,5336,5337,5338,5339,5340,5341,5342,5343,5344,5345,5346,5347, // 624
99 5348,5349,5350,5351,5352,5353,5354,5355,5356,5357,5358,5359,5360,5361,5362,5363, // 640
100 5364,5365,5366,5367,5368,5369,5370,5371,5372,5373,5374,5375,5376,5377,5378,5379, // 656
101 5380,5381, 363, 642,2787,2878,2788,2789,2316,3232,2317,3434,2011, 165,1942,3930, // 672
102 3931,3932,3933,5382,4619,5383,4620,5384,5385,5386,5387,5388,5389,5390,5391,5392, // 688
103 5393,5394,5395,5396,5397,5398,5399,5400,5401,5402,5403,5404,5405,5406,5407,5408, // 704
104 5409,5410,5411,5412,5413,5414,5415,5416,5417,5418,5419,5420,5421,5422,5423,5424, // 720
105 5425,5426,5427,5428,5429,5430,5431,5432,5433,5434,5435,5436,5437,5438,5439,5440, // 736
106 5441,5442,5443,5444,5445,5446,5447,5448,5449,5450,5451,5452,5453,5454,5455,5456, // 752
107 5457,5458,5459,5460,5461,5462,5463,5464,5465,5466,5467,5468,5469,5470,5471,5472, // 768
108 5473,5474,5475,5476,5477,5478,5479,5480,5481,5482,5483,5484,5485,5486,5487,5488, // 784
109 5489,5490,5491,5492,5493,5494,5495,5496,5497,5498,5499,5500,5501,5502,5503,5504, // 800
110 5505,5506,5507,5508,5509,5510,5511,5512,5513,5514,5515,5516,5517,5518,5519,5520, // 816
111 5521,5522,5523,5524,5525,5526,5527,5528,5529,5530,5531,5532,5533,5534,5535,5536, // 832
112 5537,5538,5539,5540,5541,5542,5543,5544,5545,5546,5547,5548,5549,5550,5551,5552, // 848
113 5553,5554,5555,5556,5557,5558,5559,5560,5561,5562,5563,5564,5565,5566,5567,5568, // 864
114 5569,5570,5571,5572,5573,5574,5575,5576,5577,5578,5579,5580,5581,5582,5583,5584, // 880
115 5585,5586,5587,5588,5589,5590,5591,5592,5593,5594,5595,5596,5597,5598,5599,5600, // 896
116 5601,5602,5603,5604,5605,5606,5607,5608,5609,5610,5611,5612,5613,5614,5615,5616, // 912
117 5617,5618,5619,5620,5621,5622,5623,5624,5625,5626,5627,5628,5629,5630,5631,5632, // 928
118 5633,5634,5635,5636,5637,5638,5639,5640,5641,5642,5643,5644,5645,5646,5647,5648, // 944
119 5649,5650,5651,5652,5653,5654,5655,5656,5657,5658,5659,5660,5661,5662,5663,5664, // 960
120 5665,5666,5667,5668,5669,5670,5671,5672,5673,5674,5675,5676,5677,5678,5679,5680, // 976
121 5681,5682,5683,5684,5685,5686,5687,5688,5689,5690,5691,5692,5693,5694,5695,5696, // 992
122 5697,5698,5699,5700,5701,5702,5703,5704,5705,5706,5707,5708,5709,5710,5711,5712, // 1008
123 5713,5714,5715,5716,5717,5718,5719,5720,5721,5722,5723,5724,5725,5726,5727,5728, // 1024
124 5729,5730,5731,5732,5733,5734,5735,5736,5737,5738,5739,5740,5741,5742,5743,5744, // 1040
125 5745,5746,5747,5748,5749,5750,5751,5752,5753,5754,5755,5756,5757,5758,5759,5760, // 1056
126 5761,5762,5763,5764,5765,5766,5767,5768,5769,5770,5771,5772,5773,5774,5775,5776, // 1072
127 5777,5778,5779,5780,5781,5782,5783,5784,5785,5786,5787,5788,5789,5790,5791,5792, // 1088
128 5793,5794,5795,5796,5797,5798,5799,5800,5801,5802,5803,5804,5805,5806,5807,5808, // 1104
129 5809,5810,5811,5812,5813,5814,5815,5816,5817,5818,5819,5820,5821,5822,5823,5824, // 1120
130 5825,5826,5827,5828,5829,5830,5831,5832,5833,5834,5835,5836,5837,5838,5839,5840, // 1136
131 5841,5842,5843,5844,5845,5846,5847,5848,5849,5850,5851,5852,5853,5854,5855,5856, // 1152
132 5857,5858,5859,5860,5861,5862,5863,5864,5865,5866,5867,5868,5869,5870,5871,5872, // 1168
133 5873,5874,5875,5876,5877,5878,5879,5880,5881,5882,5883,5884,5885,5886,5887,5888, // 1184
134 5889,5890,5891,5892,5893,5894,5895,5896,5897,5898,5899,5900,5901,5902,5903,5904, // 1200
135 5905,5906,5907,5908,5909,5910,5911,5912,5913,5914,5915,5916,5917,5918,5919,5920, // 1216
136 5921,5922,5923,5924,5925,5926,5927,5928,5929,5930,5931,5932,5933,5934,5935,5936, // 1232
137 5937,5938,5939,5940,5941,5942,5943,5944,5945,5946,5947,5948,5949,5950,5951,5952, // 1248
138 5953,5954,5955,5956,5957,5958,5959,5960,5961,5962,5963,5964,5965,5966,5967,5968, // 1264
139 5969,5970,5971,5972,5973,5974,5975,5976,5977,5978,5979,5980,5981,5982,5983,5984, // 1280
140 5985,5986,5987,5988,5989,5990,5991,5992,5993,5994,5995,5996,5997,5998,5999,6000, // 1296
141 6001,6002,6003,6004,6005,6006,6007,6008,6009,6010,6011,6012,6013,6014,6015,6016, // 1312
142 6017,6018,6019,6020,6021,6022,6023,6024,6025,6026,6027,6028,6029,6030,6031,6032, // 1328
143 6033,6034,6035,6036,6037,6038,6039,6040,6041,6042,6043,6044,6045,6046,6047,6048, // 1344
144 6049,6050,6051,6052,6053,6054,6055,6056,6057,6058,6059,6060,6061,6062,6063,6064, // 1360
145 6065,6066,6067,6068,6069,6070,6071,6072,6073,6074,6075,6076,6077,6078,6079,6080, // 1376
146 6081,6082,6083,6084,6085,6086,6087,6088,6089,6090,6091,6092,6093,6094,6095,6096, // 1392
147 6097,6098,6099,6100,6101,6102,6103,6104,6105,6106,6107,6108,6109,6110,6111,6112, // 1408
148 6113,6114,2044,2060,4621, 997,1235, 473,1186,4622, 920,3378,6115,6116, 379,1108, // 1424
149 4313,2657,2735,3934,6117,3809, 636,3233, 573,1026,3693,3435,2974,3300,2298,4105, // 1440
150 854,2937,2463, 393,2581,2417, 539, 752,1280,2750,2480, 140,1161, 440, 708,1569, // 1456
151 665,2497,1746,1291,1523,3000, 164,1603, 847,1331, 537,1997, 486, 508,1693,2418, // 1472
152 1970,2227, 878,1220, 299,1030, 969, 652,2751, 624,1137,3301,2619, 65,3302,2045, // 1488
153 1761,1859,3120,1930,3694,3516, 663,1767, 852, 835,3695, 269, 767,2826,2339,1305, // 1504
154 896,1150, 770,1616,6118, 506,1502,2075,1012,2519, 775,2520,2975,2340,2938,4314, // 1520
155 3028,2086,1224,1943,2286,6119,3072,4315,2240,1273,1987,3935,1557, 175, 597, 985, // 1536
156 3517,2419,2521,1416,3029, 585, 938,1931,1007,1052,1932,1685,6120,3379,4316,4623, // 1552
157 804, 599,3121,1333,2128,2539,1159,1554,2032,3810, 687,2033,2904, 952, 675,1467, // 1568
158 3436,6121,2241,1096,1786,2440,1543,1924, 980,1813,2228, 781,2692,1879, 728,1918, // 1584
159 3696,4624, 548,1950,4625,1809,1088,1356,3303,2522,1944, 502, 972, 373, 513,2827, // 1600
160 586,2377,2391,1003,1976,1631,6122,2464,1084, 648,1776,4626,2141, 324, 962,2012, // 1616
161 2177,2076,1384, 742,2178,1448,1173,1810, 222, 102, 301, 445, 125,2420, 662,2498, // 1632
162 277, 200,1476,1165,1068, 224,2562,1378,1446, 450,1880, 659, 791, 582,4627,2939, // 1648
163 3936,1516,1274, 555,2099,3697,1020,1389,1526,3380,1762,1723,1787,2229, 412,2114, // 1664
164 1900,2392,3518, 512,2597, 427,1925,2341,3122,1653,1686,2465,2499, 697, 330, 273, // 1680
165 380,2162, 951, 832, 780, 991,1301,3073, 965,2270,3519, 668,2523,2636,1286, 535, // 1696
166 1407, 518, 671, 957,2658,2378, 267, 611,2197,3030,6123, 248,2299, 967,1799,2356, // 1712
167 850,1418,3437,1876,1256,1480,2828,1718,6124,6125,1755,1664,2405,6126,4628,2879, // 1728
168 2829, 499,2179, 676,4629, 557,2329,2214,2090, 325,3234, 464, 811,3001, 992,2342, // 1744
169 2481,1232,1469, 303,2242, 466,1070,2163, 603,1777,2091,4630,2752,4631,2714, 322, // 1760
170 2659,1964,1768, 481,2188,1463,2330,2857,3600,2092,3031,2421,4632,2318,2070,1849, // 1776
171 2598,4633,1302,2254,1668,1701,2422,3811,2905,3032,3123,2046,4106,1763,1694,4634, // 1792
172 1604, 943,1724,1454, 917, 868,2215,1169,2940, 552,1145,1800,1228,1823,1955, 316, // 1808
173 1080,2510, 361,1807,2830,4107,2660,3381,1346,1423,1134,4108,6127, 541,1263,1229, // 1824
174 1148,2540, 545, 465,1833,2880,3438,1901,3074,2482, 816,3937, 713,1788,2500, 122, // 1840
175 1575, 195,1451,2501,1111,6128, 859, 374,1225,2243,2483,4317, 390,1033,3439,3075, // 1856
176 2524,1687, 266, 793,1440,2599, 946, 779, 802, 507, 897,1081, 528,2189,1292, 711, // 1872
177 1866,1725,1167,1640, 753, 398,2661,1053, 246, 348,4318, 137,1024,3440,1600,2077, // 1888
178 2129, 825,4319, 698, 238, 521, 187,2300,1157,2423,1641,1605,1464,1610,1097,2541, // 1904
179 1260,1436, 759,2255,1814,2150, 705,3235, 409,2563,3304, 561,3033,2005,2564, 726, // 1920
180 1956,2343,3698,4109, 949,3812,3813,3520,1669, 653,1379,2525, 881,2198, 632,2256, // 1936
181 1027, 778,1074, 733,1957, 514,1481,2466, 554,2180, 702,3938,1606,1017,1398,6129, // 1952
182 1380,3521, 921, 993,1313, 594, 449,1489,1617,1166, 768,1426,1360, 495,1794,3601, // 1968
183 1177,3602,1170,4320,2344, 476, 425,3167,4635,3168,1424, 401,2662,1171,3382,1998, // 1984
184 1089,4110, 477,3169, 474,6130,1909, 596,2831,1842, 494, 693,1051,1028,1207,3076, // 2000
185 606,2115, 727,2790,1473,1115, 743,3522, 630, 805,1532,4321,2021, 366,1057, 838, // 2016
186 684,1114,2142,4322,2050,1492,1892,1808,2271,3814,2424,1971,1447,1373,3305,1090, // 2032
187 1536,3939,3523,3306,1455,2199, 336, 369,2331,1035, 584,2393, 902, 718,2600,6131, // 2048
188 2753, 463,2151,1149,1611,2467, 715,1308,3124,1268, 343,1413,3236,1517,1347,2663, // 2064
189 2093,3940,2022,1131,1553,2100,2941,1427,3441,2942,1323,2484,6132,1980, 872,2368, // 2080
190 2441,2943, 320,2369,2116,1082, 679,1933,3941,2791,3815, 625,1143,2023, 422,2200, // 2096
191 3816,6133, 730,1695, 356,2257,1626,2301,2858,2637,1627,1778, 937, 883,2906,2693, // 2112
192 3002,1769,1086, 400,1063,1325,3307,2792,4111,3077, 456,2345,1046, 747,6134,1524, // 2128
193 884,1094,3383,1474,2164,1059, 974,1688,2181,2258,1047, 345,1665,1187, 358, 875, // 2144
194 3170, 305, 660,3524,2190,1334,1135,3171,1540,1649,2542,1527, 927, 968,2793, 885, // 2160
195 1972,1850, 482, 500,2638,1218,1109,1085,2543,1654,2034, 876, 78,2287,1482,1277, // 2176
196 861,1675,1083,1779, 724,2754, 454, 397,1132,1612,2332, 893, 672,1237, 257,2259, // 2192
197 2370, 135,3384, 337,2244, 547, 352, 340, 709,2485,1400, 788,1138,2511, 540, 772, // 2208
198 1682,2260,2272,2544,2013,1843,1902,4636,1999,1562,2288,4637,2201,1403,1533, 407, // 2224
199 576,3308,1254,2071, 978,3385, 170, 136,1201,3125,2664,3172,2394, 213, 912, 873, // 2240
200 3603,1713,2202, 699,3604,3699, 813,3442, 493, 531,1054, 468,2907,1483, 304, 281, // 2256
201 4112,1726,1252,2094, 339,2319,2130,2639, 756,1563,2944, 748, 571,2976,1588,2425, // 2272
202 2715,1851,1460,2426,1528,1392,1973,3237, 288,3309, 685,3386, 296, 892,2716,2216, // 2288
203 1570,2245, 722,1747,2217, 905,3238,1103,6135,1893,1441,1965, 251,1805,2371,3700, // 2304
204 2601,1919,1078, 75,2182,1509,1592,1270,2640,4638,2152,6136,3310,3817, 524, 706, // 2320
205 1075, 292,3818,1756,2602, 317, 98,3173,3605,3525,1844,2218,3819,2502, 814, 567, // 2336
206 385,2908,1534,6137, 534,1642,3239, 797,6138,1670,1529, 953,4323, 188,1071, 538, // 2352
207 178, 729,3240,2109,1226,1374,2000,2357,2977, 731,2468,1116,2014,2051,6139,1261, // 2368
208 1593, 803,2859,2736,3443, 556, 682, 823,1541,6140,1369,2289,1706,2794, 845, 462, // 2384
209 2603,2665,1361, 387, 162,2358,1740, 739,1770,1720,1304,1401,3241,1049, 627,1571, // 2400
210 2427,3526,1877,3942,1852,1500, 431,1910,1503, 677, 297,2795, 286,1433,1038,1198, // 2416
211 2290,1133,1596,4113,4639,2469,1510,1484,3943,6141,2442, 108, 712,4640,2372, 866, // 2432
212 3701,2755,3242,1348, 834,1945,1408,3527,2395,3243,1811, 824, 994,1179,2110,1548, // 2448
213 1453, 790,3003, 690,4324,4325,2832,2909,3820,1860,3821, 225,1748, 310, 346,1780, // 2464
214 2470, 821,1993,2717,2796, 828, 877,3528,2860,2471,1702,2165,2910,2486,1789, 453, // 2480
215 359,2291,1676, 73,1164,1461,1127,3311, 421, 604, 314,1037, 589, 116,2487, 737, // 2496
216 837,1180, 111, 244, 735,6142,2261,1861,1362, 986, 523, 418, 581,2666,3822, 103, // 2512
217 855, 503,1414,1867,2488,1091, 657,1597, 979, 605,1316,4641,1021,2443,2078,2001, // 2528
218 1209, 96, 587,2166,1032, 260,1072,2153, 173, 94, 226,3244, 819,2006,4642,4114, // 2544
219 2203, 231,1744, 782, 97,2667, 786,3387, 887, 391, 442,2219,4326,1425,6143,2694, // 2560
220 633,1544,1202, 483,2015, 592,2052,1958,2472,1655, 419, 129,4327,3444,3312,1714, // 2576
221 1257,3078,4328,1518,1098, 865,1310,1019,1885,1512,1734, 469,2444, 148, 773, 436, // 2592
222 1815,1868,1128,1055,4329,1245,2756,3445,2154,1934,1039,4643, 579,1238, 932,2320, // 2608
223 353, 205, 801, 115,2428, 944,2321,1881, 399,2565,1211, 678, 766,3944, 335,2101, // 2624
224 1459,1781,1402,3945,2737,2131,1010, 844, 981,1326,1013, 550,1816,1545,2620,1335, // 2640
225 1008, 371,2881, 936,1419,1613,3529,1456,1395,2273,1834,2604,1317,2738,2503, 416, // 2656
226 1643,4330, 806,1126, 229, 591,3946,1314,1981,1576,1837,1666, 347,1790, 977,3313, // 2672
227 764,2861,1853, 688,2429,1920,1462, 77, 595, 415,2002,3034, 798,1192,4115,6144, // 2688
228 2978,4331,3035,2695,2582,2072,2566, 430,2430,1727, 842,1396,3947,3702, 613, 377, // 2704
229 278, 236,1417,3388,3314,3174, 757,1869, 107,3530,6145,1194, 623,2262, 207,1253, // 2720
230 2167,3446,3948, 492,1117,1935, 536,1838,2757,1246,4332, 696,2095,2406,1393,1572, // 2736
231 3175,1782, 583, 190, 253,1390,2230, 830,3126,3389, 934,3245,1703,1749,2979,1870, // 2752
232 2545,1656,2204, 869,2346,4116,3176,1817, 496,1764,4644, 942,1504, 404,1903,1122, // 2768
233 1580,3606,2945,1022, 515, 372,1735, 955,2431,3036,6146,2797,1110,2302,2798, 617, // 2784
234 6147, 441, 762,1771,3447,3607,3608,1904, 840,3037, 86, 939,1385, 572,1370,2445, // 2800
235 1336, 114,3703, 898, 294, 203,3315, 703,1583,2274, 429, 961,4333,1854,1951,3390, // 2816
236 2373,3704,4334,1318,1381, 966,1911,2322,1006,1155, 309, 989, 458,2718,1795,1372, // 2832
237 1203, 252,1689,1363,3177, 517,1936, 168,1490, 562, 193,3823,1042,4117,1835, 551, // 2848
238 470,4645, 395, 489,3448,1871,1465,2583,2641, 417,1493, 279,1295, 511,1236,1119, // 2864
239 72,1231,1982,1812,3004, 871,1564, 984,3449,1667,2696,2096,4646,2347,2833,1673, // 2880
240 3609, 695,3246,2668, 807,1183,4647, 890, 388,2333,1801,1457,2911,1765,1477,1031, // 2896
241 3316,3317,1278,3391,2799,2292,2526, 163,3450,4335,2669,1404,1802,6148,2323,2407, // 2912
242 1584,1728,1494,1824,1269, 298, 909,3318,1034,1632, 375, 776,1683,2061, 291, 210, // 2928
243 1123, 809,1249,1002,2642,3038, 206,1011,2132, 144, 975, 882,1565, 342, 667, 754, // 2944
244 1442,2143,1299,2303,2062, 447, 626,2205,1221,2739,2912,1144,1214,2206,2584, 760, // 2960
245 1715, 614, 950,1281,2670,2621, 810, 577,1287,2546,4648, 242,2168, 250,2643, 691, // 2976
246 123,2644, 647, 313,1029, 689,1357,2946,1650, 216, 771,1339,1306, 808,2063, 549, // 2992
247 913,1371,2913,2914,6149,1466,1092,1174,1196,1311,2605,2396,1783,1796,3079, 406, // 3008
248 2671,2117,3949,4649, 487,1825,2220,6150,2915, 448,2348,1073,6151,2397,1707, 130, // 3024
249 900,1598, 329, 176,1959,2527,1620,6152,2275,4336,3319,1983,2191,3705,3610,2155, // 3040
250 3706,1912,1513,1614,6153,1988, 646, 392,2304,1589,3320,3039,1826,1239,1352,1340, // 3056
251 2916, 505,2567,1709,1437,2408,2547, 906,6154,2672, 384,1458,1594,1100,1329, 710, // 3072
252 423,3531,2064,2231,2622,1989,2673,1087,1882, 333, 841,3005,1296,2882,2379, 580, // 3088
253 1937,1827,1293,2585, 601, 574, 249,1772,4118,2079,1120, 645, 901,1176,1690, 795, // 3104
254 2207, 478,1434, 516,1190,1530, 761,2080, 930,1264, 355, 435,1552, 644,1791, 987, // 3120
255 220,1364,1163,1121,1538, 306,2169,1327,1222, 546,2645, 218, 241, 610,1704,3321, // 3136
256 1984,1839,1966,2528, 451,6155,2586,3707,2568, 907,3178, 254,2947, 186,1845,4650, // 3152
257 745, 432,1757, 428,1633, 888,2246,2221,2489,3611,2118,1258,1265, 956,3127,1784, // 3168
258 4337,2490, 319, 510, 119, 457,3612, 274,2035,2007,4651,1409,3128, 970,2758, 590, // 3184
259 2800, 661,2247,4652,2008,3950,1420,1549,3080,3322,3951,1651,1375,2111, 485,2491, // 3200
260 1429,1156,6156,2548,2183,1495, 831,1840,2529,2446, 501,1657, 307,1894,3247,1341, // 3216
261 666, 899,2156,1539,2549,1559, 886, 349,2208,3081,2305,1736,3824,2170,2759,1014, // 3232
262 1913,1386, 542,1397,2948, 490, 368, 716, 362, 159, 282,2569,1129,1658,1288,1750, // 3248
263 2674, 276, 649,2016, 751,1496, 658,1818,1284,1862,2209,2087,2512,3451, 622,2834, // 3264
264 376, 117,1060,2053,1208,1721,1101,1443, 247,1250,3179,1792,3952,2760,2398,3953, // 3280
265 6157,2144,3708, 446,2432,1151,2570,3452,2447,2761,2835,1210,2448,3082, 424,2222, // 3296
266 1251,2449,2119,2836, 504,1581,4338, 602, 817, 857,3825,2349,2306, 357,3826,1470, // 3312
267 1883,2883, 255, 958, 929,2917,3248, 302,4653,1050,1271,1751,2307,1952,1430,2697, // 3328
268 2719,2359, 354,3180, 777, 158,2036,4339,1659,4340,4654,2308,2949,2248,1146,2232, // 3344
269 3532,2720,1696,2623,3827,6158,3129,1550,2698,1485,1297,1428, 637, 931,2721,2145, // 3360
270 914,2550,2587, 81,2450, 612, 827,2646,1242,4655,1118,2884, 472,1855,3181,3533, // 3376
271 3534, 569,1353,2699,1244,1758,2588,4119,2009,2762,2171,3709,1312,1531,6159,1152, // 3392
272 1938, 134,1830, 471,3710,2276,1112,1535,3323,3453,3535, 982,1337,2950, 488, 826, // 3408
273 674,1058,1628,4120,2017, 522,2399, 211, 568,1367,3454, 350, 293,1872,1139,3249, // 3424
274 1399,1946,3006,1300,2360,3324, 588, 736,6160,2606, 744, 669,3536,3828,6161,1358, // 3440
275 199, 723, 848, 933, 851,1939,1505,1514,1338,1618,1831,4656,1634,3613, 443,2740, // 3456
276 3829, 717,1947, 491,1914,6162,2551,1542,4121,1025,6163,1099,1223, 198,3040,2722, // 3472
277 370, 410,1905,2589, 998,1248,3182,2380, 519,1449,4122,1710, 947, 928,1153,4341, // 3488
278 2277, 344,2624,1511, 615, 105, 161,1212,1076,1960,3130,2054,1926,1175,1906,2473, // 3504
279 414,1873,2801,6164,2309, 315,1319,3325, 318,2018,2146,2157, 963, 631, 223,4342, // 3520
280 4343,2675, 479,3711,1197,2625,3712,2676,2361,6165,4344,4123,6166,2451,3183,1886, // 3536
281 2184,1674,1330,1711,1635,1506, 799, 219,3250,3083,3954,1677,3713,3326,2081,3614, // 3552
282 1652,2073,4657,1147,3041,1752, 643,1961, 147,1974,3955,6167,1716,2037, 918,3007, // 3568
283 1994, 120,1537, 118, 609,3184,4345, 740,3455,1219, 332,1615,3830,6168,1621,2980, // 3584
284 1582, 783, 212, 553,2350,3714,1349,2433,2082,4124, 889,6169,2310,1275,1410, 973, // 3600
285 166,1320,3456,1797,1215,3185,2885,1846,2590,2763,4658, 629, 822,3008, 763, 940, // 3616
286 1990,2862, 439,2409,1566,1240,1622, 926,1282,1907,2764, 654,2210,1607, 327,1130, // 3632
287 3956,1678,1623,6170,2434,2192, 686, 608,3831,3715, 903,3957,3042,6171,2741,1522, // 3648
288 1915,1105,1555,2552,1359, 323,3251,4346,3457, 738,1354,2553,2311,2334,1828,2003, // 3664
289 3832,1753,2351,1227,6172,1887,4125,1478,6173,2410,1874,1712,1847, 520,1204,2607, // 3680
290 264,4659, 836,2677,2102, 600,4660,3833,2278,3084,6174,4347,3615,1342, 640, 532, // 3696
291 543,2608,1888,2400,2591,1009,4348,1497, 341,1737,3616,2723,1394, 529,3252,1321, // 3712
292 983,4661,1515,2120, 971,2592, 924, 287,1662,3186,4349,2700,4350,1519, 908,1948, // 3728
293 2452, 156, 796,1629,1486,2223,2055, 694,4126,1259,1036,3392,1213,2249,2742,1889, // 3744
294 1230,3958,1015, 910, 408, 559,3617,4662, 746, 725, 935,4663,3959,3009,1289, 563, // 3760
295 867,4664,3960,1567,2981,2038,2626, 988,2263,2381,4351, 143,2374, 704,1895,6175, // 3776
296 1188,3716,2088, 673,3085,2362,4352, 484,1608,1921,2765,2918, 215, 904,3618,3537, // 3792
297 894, 509, 976,3043,2701,3961,4353,2837,2982, 498,6176,6177,1102,3538,1332,3393, // 3808
298 1487,1636,1637, 233, 245,3962, 383, 650, 995,3044, 460,1520,1206,2352, 749,3327, // 3824
299 530, 700, 389,1438,1560,1773,3963,2264, 719,2951,2724,3834, 870,1832,1644,1000, // 3840
300 839,2474,3717, 197,1630,3394, 365,2886,3964,1285,2133, 734, 922, 818,1106, 732, // 3856
301 480,2083,1774,3458, 923,2279,1350, 221,3086, 85,2233,2234,3835,1585,3010,2147, // 3872
302 1387,1705,2382,1619,2475, 133, 239,2802,1991,1016,2084,2383, 411,2838,1113, 651, // 3888
303 1985,1160,3328, 990,1863,3087,1048,1276,2647, 265,2627,1599,3253,2056, 150, 638, // 3904
304 2019, 656, 853, 326,1479, 680,1439,4354,1001,1759, 413,3459,3395,2492,1431, 459, // 3920
305 4355,1125,3329,2265,1953,1450,2065,2863, 849, 351,2678,3131,3254,3255,1104,1577, // 3936
306 227,1351,1645,2453,2193,1421,2887, 812,2121, 634, 95,2435, 201,2312,4665,1646, // 3952
307 1671,2743,1601,2554,2702,2648,2280,1315,1366,2089,3132,1573,3718,3965,1729,1189, // 3968
308 328,2679,1077,1940,1136, 558,1283, 964,1195, 621,2074,1199,1743,3460,3619,1896, // 3984
309 1916,1890,3836,2952,1154,2112,1064, 862, 378,3011,2066,2113,2803,1568,2839,6178, // 4000
310 3088,2919,1941,1660,2004,1992,2194, 142, 707,1590,1708,1624,1922,1023,1836,1233, // 4016
311 1004,2313, 789, 741,3620,6179,1609,2411,1200,4127,3719,3720,4666,2057,3721, 593, // 4032
312 2840, 367,2920,1878,6180,3461,1521, 628,1168, 692,2211,2649, 300, 720,2067,2571, // 4048
313 2953,3396, 959,2504,3966,3539,3462,1977, 701,6181, 954,1043, 800, 681, 183,3722, // 4064
314 1803,1730,3540,4128,2103, 815,2314, 174, 467, 230,2454,1093,2134, 755,3541,3397, // 4080
315 1141,1162,6182,1738,2039, 270,3256,2513,1005,1647,2185,3837, 858,1679,1897,1719, // 4096
316 2954,2324,1806, 402, 670, 167,4129,1498,2158,2104, 750,6183, 915, 189,1680,1551, // 4112
317 455,4356,1501,2455, 405,1095,2955, 338,1586,1266,1819, 570, 641,1324, 237,1556, // 4128
318 2650,1388,3723,6184,1368,2384,1343,1978,3089,2436, 879,3724, 792,1191, 758,3012, // 4144
319 1411,2135,1322,4357, 240,4667,1848,3725,1574,6185, 420,3045,1546,1391, 714,4358, // 4160
320 1967, 941,1864, 863, 664, 426, 560,1731,2680,1785,2864,1949,2363, 403,3330,1415, // 4176
321 1279,2136,1697,2335, 204, 721,2097,3838, 90,6186,2085,2505, 191,3967, 124,2148, // 4192
322 1376,1798,1178,1107,1898,1405, 860,4359,1243,1272,2375,2983,1558,2456,1638, 113, // 4208
323 3621, 578,1923,2609, 880, 386,4130, 784,2186,2266,1422,2956,2172,1722, 497, 263, // 4224
324 2514,1267,2412,2610, 177,2703,3542, 774,1927,1344, 616,1432,1595,1018, 172,4360, // 4240
325 2325, 911,4361, 438,1468,3622, 794,3968,2024,2173,1681,1829,2957, 945, 895,3090, // 4256
326 575,2212,2476, 475,2401,2681, 785,2744,1745,2293,2555,1975,3133,2865, 394,4668, // 4272
327 3839, 635,4131, 639, 202,1507,2195,2766,1345,1435,2572,3726,1908,1184,1181,2457, // 4288
328 3727,3134,4362, 843,2611, 437, 916,4669, 234, 769,1884,3046,3047,3623, 833,6187, // 4304
329 1639,2250,2402,1355,1185,2010,2047, 999, 525,1732,1290,1488,2612, 948,1578,3728, // 4320
330 2413,2477,1216,2725,2159, 334,3840,1328,3624,2921,1525,4132, 564,1056, 891,4363, // 4336
331 1444,1698,2385,2251,3729,1365,2281,2235,1717,6188, 864,3841,2515, 444, 527,2767, // 4352
332 2922,3625, 544, 461,6189, 566, 209,2437,3398,2098,1065,2068,3331,3626,3257,2137, // 4368 //last 512
333
334 /***************************************************************************************
335 *Everything below is of no interest for detection purpose *
336 ***************************************************************************************
337
338 2138,2122,3730,2888,1995,1820,1044,6190,6191,6192,6193,6194,6195,6196,6197,6198, // 4384
339 6199,6200,6201,6202,6203,6204,6205,4670,6206,6207,6208,6209,6210,6211,6212,6213, // 4400
340 6214,6215,6216,6217,6218,6219,6220,6221,6222,6223,6224,6225,6226,6227,6228,6229, // 4416
341 6230,6231,6232,6233,6234,6235,6236,6237,3187,6238,6239,3969,6240,6241,6242,6243, // 4432
342 6244,4671,6245,6246,4672,6247,6248,4133,6249,6250,4364,6251,2923,2556,2613,4673, // 4448
343 4365,3970,6252,6253,6254,6255,4674,6256,6257,6258,2768,2353,4366,4675,4676,3188, // 4464
344 4367,3463,6259,4134,4677,4678,6260,2267,6261,3842,3332,4368,3543,6262,6263,6264, // 4480
345 3013,1954,1928,4135,4679,6265,6266,2478,3091,6267,4680,4369,6268,6269,1699,6270, // 4496
346 3544,4136,4681,6271,4137,6272,4370,2804,6273,6274,2593,3971,3972,4682,6275,2236, // 4512
347 4683,6276,6277,4684,6278,6279,4138,3973,4685,6280,6281,3258,6282,6283,6284,6285, // 4528
348 3974,4686,2841,3975,6286,6287,3545,6288,6289,4139,4687,4140,6290,4141,6291,4142, // 4544
349 6292,6293,3333,6294,6295,6296,4371,6297,3399,6298,6299,4372,3976,6300,6301,6302, // 4560
350 4373,6303,6304,3843,3731,6305,4688,4374,6306,6307,3259,2294,6308,3732,2530,4143, // 4576
351 6309,4689,6310,6311,6312,3048,6313,6314,4690,3733,2237,6315,6316,2282,3334,6317, // 4592
352 6318,3844,6319,6320,4691,6321,3400,4692,6322,4693,6323,3049,6324,4375,6325,3977, // 4608
353 6326,6327,6328,3546,6329,4694,3335,6330,4695,4696,6331,6332,6333,6334,4376,3978, // 4624
354 6335,4697,3979,4144,6336,3980,4698,6337,6338,6339,6340,6341,4699,4700,4701,6342, // 4640
355 6343,4702,6344,6345,4703,6346,6347,4704,6348,4705,4706,3135,6349,4707,6350,4708, // 4656
356 6351,4377,6352,4709,3734,4145,6353,2506,4710,3189,6354,3050,4711,3981,6355,3547, // 4672
357 3014,4146,4378,3735,2651,3845,3260,3136,2224,1986,6356,3401,6357,4712,2594,3627, // 4688
358 3137,2573,3736,3982,4713,3628,4714,4715,2682,3629,4716,6358,3630,4379,3631,6359, // 4704
359 6360,6361,3983,6362,6363,6364,6365,4147,3846,4717,6366,6367,3737,2842,6368,4718, // 4720
360 2628,6369,3261,6370,2386,6371,6372,3738,3984,4719,3464,4720,3402,6373,2924,3336, // 4736
361 4148,2866,6374,2805,3262,4380,2704,2069,2531,3138,2806,2984,6375,2769,6376,4721, // 4752
362 4722,3403,6377,6378,3548,6379,6380,2705,3092,1979,4149,2629,3337,2889,6381,3338, // 4768
363 4150,2557,3339,4381,6382,3190,3263,3739,6383,4151,4723,4152,2558,2574,3404,3191, // 4784
364 6384,6385,4153,6386,4724,4382,6387,6388,4383,6389,6390,4154,6391,4725,3985,6392, // 4800
365 3847,4155,6393,6394,6395,6396,6397,3465,6398,4384,6399,6400,6401,6402,6403,6404, // 4816
366 4156,6405,6406,6407,6408,2123,6409,6410,2326,3192,4726,6411,6412,6413,6414,4385, // 4832
367 4157,6415,6416,4158,6417,3093,3848,6418,3986,6419,6420,3849,6421,6422,6423,4159, // 4848
368 6424,6425,4160,6426,3740,6427,6428,6429,6430,3987,6431,4727,6432,2238,6433,6434, // 4864
369 4386,3988,6435,6436,3632,6437,6438,2843,6439,6440,6441,6442,3633,6443,2958,6444, // 4880
370 6445,3466,6446,2364,4387,3850,6447,4388,2959,3340,6448,3851,6449,4728,6450,6451, // 4896
371 3264,4729,6452,3193,6453,4389,4390,2706,3341,4730,6454,3139,6455,3194,6456,3051, // 4912
372 2124,3852,1602,4391,4161,3853,1158,3854,4162,3989,4392,3990,4731,4732,4393,2040, // 4928
373 4163,4394,3265,6457,2807,3467,3855,6458,6459,6460,3991,3468,4733,4734,6461,3140, // 4944
374 2960,6462,4735,6463,6464,6465,6466,4736,4737,4738,4739,6467,6468,4164,2403,3856, // 4960
375 6469,6470,2770,2844,6471,4740,6472,6473,6474,6475,6476,6477,6478,3195,6479,4741, // 4976
376 4395,6480,2867,6481,4742,2808,6482,2493,4165,6483,6484,6485,6486,2295,4743,6487, // 4992
377 6488,6489,3634,6490,6491,6492,6493,6494,6495,6496,2985,4744,6497,6498,4745,6499, // 5008
378 6500,2925,3141,4166,6501,6502,4746,6503,6504,4747,6505,6506,6507,2890,6508,6509, // 5024
379 6510,6511,6512,6513,6514,6515,6516,6517,6518,6519,3469,4167,6520,6521,6522,4748, // 5040
380 4396,3741,4397,4749,4398,3342,2125,4750,6523,4751,4752,4753,3052,6524,2961,4168, // 5056
381 6525,4754,6526,4755,4399,2926,4169,6527,3857,6528,4400,4170,6529,4171,6530,6531, // 5072
382 2595,6532,6533,6534,6535,3635,6536,6537,6538,6539,6540,6541,6542,4756,6543,6544, // 5088
383 6545,6546,6547,6548,4401,6549,6550,6551,6552,4402,3405,4757,4403,6553,6554,6555, // 5104
384 4172,3742,6556,6557,6558,3992,3636,6559,6560,3053,2726,6561,3549,4173,3054,4404, // 5120
385 6562,6563,3993,4405,3266,3550,2809,4406,6564,6565,6566,4758,4759,6567,3743,6568, // 5136
386 4760,3744,4761,3470,6569,6570,6571,4407,6572,3745,4174,6573,4175,2810,4176,3196, // 5152
387 4762,6574,4177,6575,6576,2494,2891,3551,6577,6578,3471,6579,4408,6580,3015,3197, // 5168
388 6581,3343,2532,3994,3858,6582,3094,3406,4409,6583,2892,4178,4763,4410,3016,4411, // 5184
389 6584,3995,3142,3017,2683,6585,4179,6586,6587,4764,4412,6588,6589,4413,6590,2986, // 5200
390 6591,2962,3552,6592,2963,3472,6593,6594,4180,4765,6595,6596,2225,3267,4414,6597, // 5216
391 3407,3637,4766,6598,6599,3198,6600,4415,6601,3859,3199,6602,3473,4767,2811,4416, // 5232
392 1856,3268,3200,2575,3996,3997,3201,4417,6603,3095,2927,6604,3143,6605,2268,6606, // 5248
393 3998,3860,3096,2771,6607,6608,3638,2495,4768,6609,3861,6610,3269,2745,4769,4181, // 5264
394 3553,6611,2845,3270,6612,6613,6614,3862,6615,6616,4770,4771,6617,3474,3999,4418, // 5280
395 4419,6618,3639,3344,6619,4772,4182,6620,2126,6621,6622,6623,4420,4773,6624,3018, // 5296
396 6625,4774,3554,6626,4183,2025,3746,6627,4184,2707,6628,4421,4422,3097,1775,4185, // 5312
397 3555,6629,6630,2868,6631,6632,4423,6633,6634,4424,2414,2533,2928,6635,4186,2387, // 5328
398 6636,4775,6637,4187,6638,1891,4425,3202,3203,6639,6640,4776,6641,3345,6642,6643, // 5344
399 3640,6644,3475,3346,3641,4000,6645,3144,6646,3098,2812,4188,3642,3204,6647,3863, // 5360
400 3476,6648,3864,6649,4426,4001,6650,6651,6652,2576,6653,4189,4777,6654,6655,6656, // 5376
401 2846,6657,3477,3205,4002,6658,4003,6659,3347,2252,6660,6661,6662,4778,6663,6664, // 5392
402 6665,6666,6667,6668,6669,4779,4780,2048,6670,3478,3099,6671,3556,3747,4004,6672, // 5408
403 6673,6674,3145,4005,3748,6675,6676,6677,6678,6679,3408,6680,6681,6682,6683,3206, // 5424
404 3207,6684,6685,4781,4427,6686,4782,4783,4784,6687,6688,6689,4190,6690,6691,3479, // 5440
405 6692,2746,6693,4428,6694,6695,6696,6697,6698,6699,4785,6700,6701,3208,2727,6702, // 5456
406 3146,6703,6704,3409,2196,6705,4429,6706,6707,6708,2534,1996,6709,6710,6711,2747, // 5472
407 6712,6713,6714,4786,3643,6715,4430,4431,6716,3557,6717,4432,4433,6718,6719,6720, // 5488
408 6721,3749,6722,4006,4787,6723,6724,3644,4788,4434,6725,6726,4789,2772,6727,6728, // 5504
409 6729,6730,6731,2708,3865,2813,4435,6732,6733,4790,4791,3480,6734,6735,6736,6737, // 5520
410 4436,3348,6738,3410,4007,6739,6740,4008,6741,6742,4792,3411,4191,6743,6744,6745, // 5536
411 6746,6747,3866,6748,3750,6749,6750,6751,6752,6753,6754,6755,3867,6756,4009,6757, // 5552
412 4793,4794,6758,2814,2987,6759,6760,6761,4437,6762,6763,6764,6765,3645,6766,6767, // 5568
413 3481,4192,6768,3751,6769,6770,2174,6771,3868,3752,6772,6773,6774,4193,4795,4438, // 5584
414 3558,4796,4439,6775,4797,6776,6777,4798,6778,4799,3559,4800,6779,6780,6781,3482, // 5600
415 6782,2893,6783,6784,4194,4801,4010,6785,6786,4440,6787,4011,6788,6789,6790,6791, // 5616
416 6792,6793,4802,6794,6795,6796,4012,6797,6798,6799,6800,3349,4803,3483,6801,4804, // 5632
417 4195,6802,4013,6803,6804,4196,6805,4014,4015,6806,2847,3271,2848,6807,3484,6808, // 5648
418 6809,6810,4441,6811,4442,4197,4443,3272,4805,6812,3412,4016,1579,6813,6814,4017, // 5664
419 6815,3869,6816,2964,6817,4806,6818,6819,4018,3646,6820,6821,4807,4019,4020,6822, // 5680
420 6823,3560,6824,6825,4021,4444,6826,4198,6827,6828,4445,6829,6830,4199,4808,6831, // 5696
421 6832,6833,3870,3019,2458,6834,3753,3413,3350,6835,4809,3871,4810,3561,4446,6836, // 5712
422 6837,4447,4811,4812,6838,2459,4448,6839,4449,6840,6841,4022,3872,6842,4813,4814, // 5728
423 6843,6844,4815,4200,4201,4202,6845,4023,6846,6847,4450,3562,3873,6848,6849,4816, // 5744
424 4817,6850,4451,4818,2139,6851,3563,6852,6853,3351,6854,6855,3352,4024,2709,3414, // 5760
425 4203,4452,6856,4204,6857,6858,3874,3875,6859,6860,4819,6861,6862,6863,6864,4453, // 5776
426 3647,6865,6866,4820,6867,6868,6869,6870,4454,6871,2869,6872,6873,4821,6874,3754, // 5792
427 6875,4822,4205,6876,6877,6878,3648,4206,4455,6879,4823,6880,4824,3876,6881,3055, // 5808
428 4207,6882,3415,6883,6884,6885,4208,4209,6886,4210,3353,6887,3354,3564,3209,3485, // 5824
429 2652,6888,2728,6889,3210,3755,6890,4025,4456,6891,4825,6892,6893,6894,6895,4211, // 5840
430 6896,6897,6898,4826,6899,6900,4212,6901,4827,6902,2773,3565,6903,4828,6904,6905, // 5856
431 6906,6907,3649,3650,6908,2849,3566,6909,3567,3100,6910,6911,6912,6913,6914,6915, // 5872
432 4026,6916,3355,4829,3056,4457,3756,6917,3651,6918,4213,3652,2870,6919,4458,6920, // 5888
433 2438,6921,6922,3757,2774,4830,6923,3356,4831,4832,6924,4833,4459,3653,2507,6925, // 5904
434 4834,2535,6926,6927,3273,4027,3147,6928,3568,6929,6930,6931,4460,6932,3877,4461, // 5920
435 2729,3654,6933,6934,6935,6936,2175,4835,2630,4214,4028,4462,4836,4215,6937,3148, // 5936
436 4216,4463,4837,4838,4217,6938,6939,2850,4839,6940,4464,6941,6942,6943,4840,6944, // 5952
437 4218,3274,4465,6945,6946,2710,6947,4841,4466,6948,6949,2894,6950,6951,4842,6952, // 5968
438 4219,3057,2871,6953,6954,6955,6956,4467,6957,2711,6958,6959,6960,3275,3101,4843, // 5984
439 6961,3357,3569,6962,4844,6963,6964,4468,4845,3570,6965,3102,4846,3758,6966,4847, // 6000
440 3878,4848,4849,4029,6967,2929,3879,4850,4851,6968,6969,1733,6970,4220,6971,6972, // 6016
441 6973,6974,6975,6976,4852,6977,6978,6979,6980,6981,6982,3759,6983,6984,6985,3486, // 6032
442 3487,6986,3488,3416,6987,6988,6989,6990,6991,6992,6993,6994,6995,6996,6997,4853, // 6048
443 6998,6999,4030,7000,7001,3211,7002,7003,4221,7004,7005,3571,4031,7006,3572,7007, // 6064
444 2614,4854,2577,7008,7009,2965,3655,3656,4855,2775,3489,3880,4222,4856,3881,4032, // 6080
445 3882,3657,2730,3490,4857,7010,3149,7011,4469,4858,2496,3491,4859,2283,7012,7013, // 6096
446 7014,2365,4860,4470,7015,7016,3760,7017,7018,4223,1917,7019,7020,7021,4471,7022, // 6112
447 2776,4472,7023,7024,7025,7026,4033,7027,3573,4224,4861,4034,4862,7028,7029,1929, // 6128
448 3883,4035,7030,4473,3058,7031,2536,3761,3884,7032,4036,7033,2966,2895,1968,4474, // 6144
449 3276,4225,3417,3492,4226,2105,7034,7035,1754,2596,3762,4227,4863,4475,3763,4864, // 6160
450 3764,2615,2777,3103,3765,3658,3418,4865,2296,3766,2815,7036,7037,7038,3574,2872, // 6176
451 3277,4476,7039,4037,4477,7040,7041,4038,7042,7043,7044,7045,7046,7047,2537,7048, // 6192
452 7049,7050,7051,7052,7053,7054,4478,7055,7056,3767,3659,4228,3575,7057,7058,4229, // 6208
453 7059,7060,7061,3660,7062,3212,7063,3885,4039,2460,7064,7065,7066,7067,7068,7069, // 6224
454 7070,7071,7072,7073,7074,4866,3768,4867,7075,7076,7077,7078,4868,3358,3278,2653, // 6240
455 7079,7080,4479,3886,7081,7082,4869,7083,7084,7085,7086,7087,7088,2538,7089,7090, // 6256
456 7091,4040,3150,3769,4870,4041,2896,3359,4230,2930,7092,3279,7093,2967,4480,3213, // 6272
457 4481,3661,7094,7095,7096,7097,7098,7099,7100,7101,7102,2461,3770,7103,7104,4231, // 6288
458 3151,7105,7106,7107,4042,3662,7108,7109,4871,3663,4872,4043,3059,7110,7111,7112, // 6304
459 3493,2988,7113,4873,7114,7115,7116,3771,4874,7117,7118,4232,4875,7119,3576,2336, // 6320
460 4876,7120,4233,3419,4044,4877,4878,4482,4483,4879,4484,4234,7121,3772,4880,1045, // 6336
461 3280,3664,4881,4882,7122,7123,7124,7125,4883,7126,2778,7127,4485,4486,7128,4884, // 6352
462 3214,3887,7129,7130,3215,7131,4885,4045,7132,7133,4046,7134,7135,7136,7137,7138, // 6368
463 7139,7140,7141,7142,7143,4235,7144,4886,7145,7146,7147,4887,7148,7149,7150,4487, // 6384
464 4047,4488,7151,7152,4888,4048,2989,3888,7153,3665,7154,4049,7155,7156,7157,7158, // 6400
465 7159,7160,2931,4889,4890,4489,7161,2631,3889,4236,2779,7162,7163,4891,7164,3060, // 6416
466 7165,1672,4892,7166,4893,4237,3281,4894,7167,7168,3666,7169,3494,7170,7171,4050, // 6432
467 7172,7173,3104,3360,3420,4490,4051,2684,4052,7174,4053,7175,7176,7177,2253,4054, // 6448
468 7178,7179,4895,7180,3152,3890,3153,4491,3216,7181,7182,7183,2968,4238,4492,4055, // 6464
469 7184,2990,7185,2479,7186,7187,4493,7188,7189,7190,7191,7192,4896,7193,4897,2969, // 6480
470 4494,4898,7194,3495,7195,7196,4899,4495,7197,3105,2731,7198,4900,7199,7200,7201, // 6496
471 4056,7202,3361,7203,7204,4496,4901,4902,7205,4497,7206,7207,2315,4903,7208,4904, // 6512
472 7209,4905,2851,7210,7211,3577,7212,3578,4906,7213,4057,3667,4907,7214,4058,2354, // 6528
473 3891,2376,3217,3773,7215,7216,7217,7218,7219,4498,7220,4908,3282,2685,7221,3496, // 6544
474 4909,2632,3154,4910,7222,2337,7223,4911,7224,7225,7226,4912,4913,3283,4239,4499, // 6560
475 7227,2816,7228,7229,7230,7231,7232,7233,7234,4914,4500,4501,7235,7236,7237,2686, // 6576
476 7238,4915,7239,2897,4502,7240,4503,7241,2516,7242,4504,3362,3218,7243,7244,7245, // 6592
477 4916,7246,7247,4505,3363,7248,7249,7250,7251,3774,4506,7252,7253,4917,7254,7255, // 6608
478 3284,2991,4918,4919,3219,3892,4920,3106,3497,4921,7256,7257,7258,4922,7259,4923, // 6624
479 3364,4507,4508,4059,7260,4240,3498,7261,7262,4924,7263,2992,3893,4060,3220,7264, // 6640
480 7265,7266,7267,7268,7269,4509,3775,7270,2817,7271,4061,4925,4510,3776,7272,4241, // 6656
481 4511,3285,7273,7274,3499,7275,7276,7277,4062,4512,4926,7278,3107,3894,7279,7280, // 6672
482 4927,7281,4513,7282,7283,3668,7284,7285,4242,4514,4243,7286,2058,4515,4928,4929, // 6688
483 4516,7287,3286,4244,7288,4517,7289,7290,7291,3669,7292,7293,4930,4931,4932,2355, // 6704
484 4933,7294,2633,4518,7295,4245,7296,7297,4519,7298,7299,4520,4521,4934,7300,4246, // 6720
485 4522,7301,7302,7303,3579,7304,4247,4935,7305,4936,7306,7307,7308,7309,3777,7310, // 6736
486 4523,7311,7312,7313,4248,3580,7314,4524,3778,4249,7315,3581,7316,3287,7317,3221, // 6752
487 7318,4937,7319,7320,7321,7322,7323,7324,4938,4939,7325,4525,7326,7327,7328,4063, // 6768
488 7329,7330,4940,7331,7332,4941,7333,4526,7334,3500,2780,1741,4942,2026,1742,7335, // 6784
489 7336,3582,4527,2388,7337,7338,7339,4528,7340,4250,4943,7341,7342,7343,4944,7344, // 6800
490 7345,7346,3020,7347,4945,7348,7349,7350,7351,3895,7352,3896,4064,3897,7353,7354, // 6816
491 7355,4251,7356,7357,3898,7358,3779,7359,3780,3288,7360,7361,4529,7362,4946,4530, // 6832
492 2027,7363,3899,4531,4947,3222,3583,7364,4948,7365,7366,7367,7368,4949,3501,4950, // 6848
493 3781,4951,4532,7369,2517,4952,4252,4953,3155,7370,4954,4955,4253,2518,4533,7371, // 6864
494 7372,2712,4254,7373,7374,7375,3670,4956,3671,7376,2389,3502,4065,7377,2338,7378, // 6880
495 7379,7380,7381,3061,7382,4957,7383,7384,7385,7386,4958,4534,7387,7388,2993,7389, // 6896
496 3062,7390,4959,7391,7392,7393,4960,3108,4961,7394,4535,7395,4962,3421,4536,7396, // 6912
497 4963,7397,4964,1857,7398,4965,7399,7400,2176,3584,4966,7401,7402,3422,4537,3900, // 6928
498 3585,7403,3782,7404,2852,7405,7406,7407,4538,3783,2654,3423,4967,4539,7408,3784, // 6944
499 3586,2853,4540,4541,7409,3901,7410,3902,7411,7412,3785,3109,2327,3903,7413,7414, // 6960
500 2970,4066,2932,7415,7416,7417,3904,3672,3424,7418,4542,4543,4544,7419,4968,7420, // 6976
501 7421,4255,7422,7423,7424,7425,7426,4067,7427,3673,3365,4545,7428,3110,2559,3674, // 6992
502 7429,7430,3156,7431,7432,3503,7433,3425,4546,7434,3063,2873,7435,3223,4969,4547, // 7008
503 4548,2898,4256,4068,7436,4069,3587,3786,2933,3787,4257,4970,4971,3788,7437,4972, // 7024
504 3064,7438,4549,7439,7440,7441,7442,7443,4973,3905,7444,2874,7445,7446,7447,7448, // 7040
505 3021,7449,4550,3906,3588,4974,7450,7451,3789,3675,7452,2578,7453,4070,7454,7455, // 7056
506 7456,4258,3676,7457,4975,7458,4976,4259,3790,3504,2634,4977,3677,4551,4260,7459, // 7072
507 7460,7461,7462,3907,4261,4978,7463,7464,7465,7466,4979,4980,7467,7468,2213,4262, // 7088
508 7469,7470,7471,3678,4981,7472,2439,7473,4263,3224,3289,7474,3908,2415,4982,7475, // 7104
509 4264,7476,4983,2655,7477,7478,2732,4552,2854,2875,7479,7480,4265,7481,4553,4984, // 7120
510 7482,7483,4266,7484,3679,3366,3680,2818,2781,2782,3367,3589,4554,3065,7485,4071, // 7136
511 2899,7486,7487,3157,2462,4072,4555,4073,4985,4986,3111,4267,2687,3368,4556,4074, // 7152
512 3791,4268,7488,3909,2783,7489,2656,1962,3158,4557,4987,1963,3159,3160,7490,3112, // 7168
513 4988,4989,3022,4990,4991,3792,2855,7491,7492,2971,4558,7493,7494,4992,7495,7496, // 7184
514 7497,7498,4993,7499,3426,4559,4994,7500,3681,4560,4269,4270,3910,7501,4075,4995, // 7200
515 4271,7502,7503,4076,7504,4996,7505,3225,4997,4272,4077,2819,3023,7506,7507,2733, // 7216
516 4561,7508,4562,7509,3369,3793,7510,3590,2508,7511,7512,4273,3113,2994,2616,7513, // 7232
517 7514,7515,7516,7517,7518,2820,3911,4078,2748,7519,7520,4563,4998,7521,7522,7523, // 7248
518 7524,4999,4274,7525,4564,3682,2239,4079,4565,7526,7527,7528,7529,5000,7530,7531, // 7264
519 5001,4275,3794,7532,7533,7534,3066,5002,4566,3161,7535,7536,4080,7537,3162,7538, // 7280
520 7539,4567,7540,7541,7542,7543,7544,7545,5003,7546,4568,7547,7548,7549,7550,7551, // 7296
521 7552,7553,7554,7555,7556,5004,7557,7558,7559,5005,7560,3795,7561,4569,7562,7563, // 7312
522 7564,2821,3796,4276,4277,4081,7565,2876,7566,5006,7567,7568,2900,7569,3797,3912, // 7328
523 7570,7571,7572,4278,7573,7574,7575,5007,7576,7577,5008,7578,7579,4279,2934,7580, // 7344
524 7581,5009,7582,4570,7583,4280,7584,7585,7586,4571,4572,3913,7587,4573,3505,7588, // 7360
525 5010,7589,7590,7591,7592,3798,4574,7593,7594,5011,7595,4281,7596,7597,7598,4282, // 7376
526 5012,7599,7600,5013,3163,7601,5014,7602,3914,7603,7604,2734,4575,4576,4577,7605, // 7392
527 7606,7607,7608,7609,3506,5015,4578,7610,4082,7611,2822,2901,2579,3683,3024,4579, // 7408
528 3507,7612,4580,7613,3226,3799,5016,7614,7615,7616,7617,7618,7619,7620,2995,3290, // 7424
529 7621,4083,7622,5017,7623,7624,7625,7626,7627,4581,3915,7628,3291,7629,5018,7630, // 7440
530 7631,7632,7633,4084,7634,7635,3427,3800,7636,7637,4582,7638,5019,4583,5020,7639, // 7456
531 3916,7640,3801,5021,4584,4283,7641,7642,3428,3591,2269,7643,2617,7644,4585,3592, // 7472
532 7645,4586,2902,7646,7647,3227,5022,7648,4587,7649,4284,7650,7651,7652,4588,2284, // 7488
533 7653,5023,7654,7655,7656,4589,5024,3802,7657,7658,5025,3508,4590,7659,7660,7661, // 7504
534 1969,5026,7662,7663,3684,1821,2688,7664,2028,2509,4285,7665,2823,1841,7666,2689, // 7520
535 3114,7667,3917,4085,2160,5027,5028,2972,7668,5029,7669,7670,7671,3593,4086,7672, // 7536
536 4591,4087,5030,3803,7673,7674,7675,7676,7677,7678,7679,4286,2366,4592,4593,3067, // 7552
537 2328,7680,7681,4594,3594,3918,2029,4287,7682,5031,3919,3370,4288,4595,2856,7683, // 7568
538 3509,7684,7685,5032,5033,7686,7687,3804,2784,7688,7689,7690,7691,3371,7692,7693, // 7584
539 2877,5034,7694,7695,3920,4289,4088,7696,7697,7698,5035,7699,5036,4290,5037,5038, // 7600
540 5039,7700,7701,7702,5040,5041,3228,7703,1760,7704,5042,3229,4596,2106,4089,7705, // 7616
541 4597,2824,5043,2107,3372,7706,4291,4090,5044,7707,4091,7708,5045,3025,3805,4598, // 7632
542 4292,4293,4294,3373,7709,4599,7710,5046,7711,7712,5047,5048,3806,7713,7714,7715, // 7648
543 5049,7716,7717,7718,7719,4600,5050,7720,7721,7722,5051,7723,4295,3429,7724,7725, // 7664
544 7726,7727,3921,7728,3292,5052,4092,7729,7730,7731,7732,7733,7734,7735,5053,5054, // 7680
545 7736,7737,7738,7739,3922,3685,7740,7741,7742,7743,2635,5055,7744,5056,4601,7745, // 7696
546 7746,2560,7747,7748,7749,7750,3923,7751,7752,7753,7754,7755,4296,2903,7756,7757, // 7712
547 7758,7759,7760,3924,7761,5057,4297,7762,7763,5058,4298,7764,4093,7765,7766,5059, // 7728
548 3925,7767,7768,7769,7770,7771,7772,7773,7774,7775,7776,3595,7777,4299,5060,4094, // 7744
549 7778,3293,5061,7779,7780,4300,7781,7782,4602,7783,3596,7784,7785,3430,2367,7786, // 7760
550 3164,5062,5063,4301,7787,7788,4095,5064,5065,7789,3374,3115,7790,7791,7792,7793, // 7776
551 7794,7795,7796,3597,4603,7797,7798,3686,3116,3807,5066,7799,7800,5067,7801,7802, // 7792
552 4604,4302,5068,4303,4096,7803,7804,3294,7805,7806,5069,4605,2690,7807,3026,7808, // 7808
553 7809,7810,7811,7812,7813,7814,7815,7816,7817,7818,7819,7820,7821,7822,7823,7824, // 7824
554 7825,7826,7827,7828,7829,7830,7831,7832,7833,7834,7835,7836,7837,7838,7839,7840, // 7840
555 7841,7842,7843,7844,7845,7846,7847,7848,7849,7850,7851,7852,7853,7854,7855,7856, // 7856
556 7857,7858,7859,7860,7861,7862,7863,7864,7865,7866,7867,7868,7869,7870,7871,7872, // 7872
557 7873,7874,7875,7876,7877,7878,7879,7880,7881,7882,7883,7884,7885,7886,7887,7888, // 7888
558 7889,7890,7891,7892,7893,7894,7895,7896,7897,7898,7899,7900,7901,7902,7903,7904, // 7904
559 7905,7906,7907,7908,7909,7910,7911,7912,7913,7914,7915,7916,7917,7918,7919,7920, // 7920
560 7921,7922,7923,7924,3926,7925,7926,7927,7928,7929,7930,7931,7932,7933,7934,7935, // 7936
561 7936,7937,7938,7939,7940,7941,7942,7943,7944,7945,7946,7947,7948,7949,7950,7951, // 7952
562 7952,7953,7954,7955,7956,7957,7958,7959,7960,7961,7962,7963,7964,7965,7966,7967, // 7968
563 7968,7969,7970,7971,7972,7973,7974,7975,7976,7977,7978,7979,7980,7981,7982,7983, // 7984
564 7984,7985,7986,7987,7988,7989,7990,7991,7992,7993,7994,7995,7996,7997,7998,7999, // 8000
565 8000,8001,8002,8003,8004,8005,8006,8007,8008,8009,8010,8011,8012,8013,8014,8015, // 8016
566 8016,8017,8018,8019,8020,8021,8022,8023,8024,8025,8026,8027,8028,8029,8030,8031, // 8032
567 8032,8033,8034,8035,8036,8037,8038,8039,8040,8041,8042,8043,8044,8045,8046,8047, // 8048
568 8048,8049,8050,8051,8052,8053,8054,8055,8056,8057,8058,8059,8060,8061,8062,8063, // 8064
569 8064,8065,8066,8067,8068,8069,8070,8071,8072,8073,8074,8075,8076,8077,8078,8079, // 8080
570 8080,8081,8082,8083,8084,8085,8086,8087,8088,8089,8090,8091,8092,8093,8094,8095, // 8096
571 8096,8097,8098,8099,8100,8101,8102,8103,8104,8105,8106,8107,8108,8109,8110,8111, // 8112
572 8112,8113,8114,8115,8116,8117,8118,8119,8120,8121,8122,8123,8124,8125,8126,8127, // 8128
573 8128,8129,8130,8131,8132,8133,8134,8135,8136,8137,8138,8139,8140,8141,8142,8143, // 8144
574 8144,8145,8146,8147,8148,8149,8150,8151,8152,8153,8154,8155,8156,8157,8158,8159, // 8160
575 8160,8161,8162,8163,8164,8165,8166,8167,8168,8169,8170,8171,8172,8173,8174,8175, // 8176
576 8176,8177,8178,8179,8180,8181,8182,8183,8184,8185,8186,8187,8188,8189,8190,8191, // 8192
577 8192,8193,8194,8195,8196,8197,8198,8199,8200,8201,8202,8203,8204,8205,8206,8207, // 8208
578 8208,8209,8210,8211,8212,8213,8214,8215,8216,8217,8218,8219,8220,8221,8222,8223, // 8224
579 8224,8225,8226,8227,8228,8229,8230,8231,8232,8233,8234,8235,8236,8237,8238,8239, // 8240
580 8240,8241,8242,8243,8244,8245,8246,8247,8248,8249,8250,8251,8252,8253,8254,8255, // 8256
581 8256,8257,8258,8259,8260,8261,8262,8263,8264,8265,8266,8267,8268,8269,8270,8271, // 8272
582 ****************************************************************************************/
583 };
584 }
+79
-0
src/main/java/org/mozilla/universalchardet/prober/distributionanalysis/SJISDistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public class SJISDistributionAnalysis extends JISDistributionAnalysis {
40 ////////////////////////////////////////////////////////////////
41 // constants
42 ////////////////////////////////////////////////////////////////
43 public static final int HIGHBYTE_BEGIN_1 = 0x81;
44 public static final int HIGHBYTE_END_1 = 0x9F;
45 public static final int HIGHBYTE_BEGIN_2 = 0xE0;
46 public static final int HIGHBYTE_END_2 = 0xEF;
47 public static final int LOWBYTE_BEGIN_1 = 0x40;
48 public static final int LOWBYTE_BEGIN_2 = 0x80;
49
50
51 ////////////////////////////////////////////////////////////////
52 // methods
53 ////////////////////////////////////////////////////////////////
54 public SJISDistributionAnalysis() {
55 super();
56 }
57
58 @Override
59 protected int getOrder(final byte[] buf, int offset) {
60 int order = -1;
61
62 int highbyte = buf[offset] & 0xFF;
63 if (highbyte >= HIGHBYTE_BEGIN_1 && highbyte <= HIGHBYTE_END_1) {
64 order = 188 * (highbyte - HIGHBYTE_BEGIN_1);
65 } else if (highbyte >= HIGHBYTE_BEGIN_2 && highbyte <= HIGHBYTE_END_2) {
66 order = 188 * (highbyte - HIGHBYTE_BEGIN_2 + 31);
67 } else {
68 return -1;
69 }
70 int lowbyte = buf[offset+1] & 0xFF;
71 order += lowbyte - LOWBYTE_BEGIN_1;
72 if (lowbyte >= LOWBYTE_BEGIN_2) {
73 --order;
74 }
75
76 return order;
77 }
78 }
+193
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/BulgarianModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 public class BulgarianModel extends SequenceModel {
40 ////////////////////////////////////////////////////////////////
41 // constants
42 ////////////////////////////////////////////////////////////////
43 public static final float TYPICAL_POSITIVE_RATIO = 0.969392f;
44
45
46 ////////////////////////////////////////////////////////////////
47 // methods
48 ////////////////////////////////////////////////////////////////
49 public BulgarianModel(short[] charToOrderMap, String name) {
50 super(
51 charToOrderMap,
52 bulgarianLangModel,
53 TYPICAL_POSITIVE_RATIO,
54 false,
55 name);
56 }
57
58
59 ////////////////////////////////////////////////////////////////
60 // constants continued
61 ////////////////////////////////////////////////////////////////
62 private static final byte[] bulgarianLangModel = new byte[] {
63 0,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,3,3,3,3,3,3,3,2,3,3,3,3,3,
64 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,0,3,3,3,2,2,3,2,2,1,2,2,
65 3,1,3,3,2,3,3,3,3,3,3,3,3,3,3,3,3,0,3,3,3,3,3,3,3,3,3,3,0,3,0,1,
66 0,0,0,0,0,0,0,0,0,0,1,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
67 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,2,3,3,3,3,3,3,3,3,0,3,1,0,
68 0,1,0,0,0,0,0,0,0,0,1,1,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
69 3,2,2,2,3,3,3,3,3,3,3,3,3,3,3,3,3,1,3,2,3,3,3,3,3,3,3,3,0,3,0,0,
70 0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
71 3,2,3,3,2,3,3,3,3,3,3,3,3,3,3,3,3,1,3,2,3,3,3,3,3,3,3,3,0,3,0,0,
72 0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
73 3,3,3,3,3,3,3,3,3,3,3,2,3,2,2,1,3,3,3,3,2,2,2,1,1,2,0,1,0,1,0,0,
74 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,1,
75 3,3,3,3,3,3,3,2,3,2,2,3,3,1,1,2,3,3,2,3,3,3,3,2,1,2,0,2,0,3,0,0,
76 0,0,0,0,0,0,0,1,0,0,2,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,1,
77 3,3,3,3,3,3,3,1,3,3,3,3,3,2,3,2,3,3,3,3,3,2,3,3,1,3,0,3,0,2,0,0,
78 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
79 3,3,3,3,3,3,3,3,1,3,3,2,3,3,3,1,3,3,2,3,2,2,2,0,0,2,0,2,0,2,0,0,
80 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,1,
81 3,3,3,3,3,3,3,3,3,0,3,3,3,2,2,3,3,3,1,2,2,3,2,1,1,2,0,2,0,0,0,0,
82 1,0,0,0,0,0,0,0,0,0,2,0,0,1,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
83 3,3,3,3,3,3,3,2,3,3,1,2,3,2,2,2,3,3,3,3,3,2,2,3,1,2,0,2,1,2,0,0,
84 0,0,0,0,0,0,0,0,0,0,3,0,0,1,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,1,
85 3,3,3,3,3,1,3,3,3,3,3,2,3,3,3,2,3,3,2,3,2,2,2,3,1,2,0,1,0,1,0,0,
86 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
87 3,3,3,3,3,3,3,3,3,3,3,1,1,1,2,2,1,3,1,3,2,2,3,0,0,1,0,1,0,1,0,0,
88 0,0,0,1,0,0,0,0,1,0,2,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
89 3,3,3,3,3,2,2,3,2,2,3,1,2,1,1,1,2,3,1,3,1,2,2,0,1,1,1,1,0,1,0,0,
90 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
91 3,3,3,3,3,1,3,2,2,3,3,1,2,3,1,1,3,3,3,3,1,2,2,1,1,1,0,2,0,2,0,1,
92 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
93 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,1,2,2,3,3,3,2,2,1,1,2,0,2,0,1,0,0,
94 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
95 3,0,1,2,1,3,3,2,3,3,3,3,3,2,3,2,1,0,3,1,2,1,2,1,2,3,2,1,0,1,0,0,
96 0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
97 1,1,1,2,3,3,3,3,3,3,3,3,3,3,3,3,0,0,3,1,3,3,2,3,3,2,2,2,0,1,0,0,
98 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
99 2,3,3,3,3,0,3,3,3,3,3,2,1,1,2,1,3,3,0,3,1,1,1,1,3,2,0,1,0,0,0,0,
100 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
101 3,3,2,2,2,3,3,3,3,3,3,3,3,3,3,3,1,1,3,1,3,3,2,3,2,2,2,3,0,2,0,0,
102 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
103 3,3,3,3,3,2,3,3,2,2,3,2,1,1,1,1,1,3,1,3,1,1,0,0,0,1,0,0,0,1,0,0,
104 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,
105 3,3,3,3,3,2,3,2,0,3,2,0,3,0,2,0,0,2,1,3,1,0,0,1,0,0,0,1,0,0,0,0,
106 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
107 3,3,3,3,2,1,1,1,1,2,1,1,2,1,1,1,2,2,1,2,1,1,1,0,1,1,0,1,0,1,0,0,
108 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
109 3,3,3,3,2,1,3,1,1,2,1,3,2,1,1,0,1,2,3,2,1,1,1,0,0,0,0,0,0,0,0,0,
110 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
111 2,3,3,3,3,2,2,1,0,1,0,0,1,0,0,0,2,1,0,3,0,0,1,0,0,0,0,0,0,0,0,0,
112 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
113 3,3,3,2,3,2,3,3,1,3,2,1,1,1,2,1,1,2,1,3,0,1,0,0,0,1,0,0,0,0,0,0,
114 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
115 3,1,1,2,2,3,3,2,3,2,2,2,3,1,2,2,1,1,2,1,1,2,2,0,1,1,0,1,0,2,0,0,
116 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
117 3,3,3,3,2,1,3,1,0,2,2,1,3,2,1,0,0,2,0,2,0,1,0,0,0,0,0,0,0,1,0,0,
118 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
119 3,3,3,3,3,3,1,2,0,2,3,1,2,3,2,0,1,3,1,2,1,1,1,0,0,1,0,0,2,2,2,3,
120 2,2,2,2,1,2,1,1,2,2,1,1,2,0,1,1,1,0,0,1,1,0,0,1,1,0,0,0,1,1,0,1,
121 3,3,3,3,3,2,1,2,2,1,2,0,2,0,1,0,1,2,1,2,1,1,0,0,0,1,0,1,0,0,0,0,
122 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,1,
123 3,3,2,3,3,1,1,3,1,0,3,2,1,0,0,0,1,2,0,2,0,1,0,0,0,1,0,1,2,1,2,2,
124 1,1,1,1,1,1,1,2,2,2,1,1,1,1,1,1,1,0,1,2,1,1,1,0,0,0,0,0,1,1,0,0,
125 3,1,0,1,0,2,3,2,2,2,3,2,2,2,2,2,1,0,2,1,2,1,1,1,0,1,2,1,2,2,2,1,
126 1,1,2,2,2,2,1,2,1,1,0,1,2,1,2,2,2,1,1,1,0,1,1,1,1,2,0,1,0,0,0,0,
127 2,3,2,3,3,0,0,2,1,0,2,1,0,0,0,0,2,3,0,2,0,0,0,0,0,1,0,0,2,0,1,2,
128 2,1,2,1,2,2,1,1,1,2,1,1,1,0,1,2,2,1,1,1,1,1,0,1,1,1,0,0,1,2,0,0,
129 3,3,2,2,3,0,2,3,1,1,2,0,0,0,1,0,0,2,0,2,0,0,0,1,0,1,0,1,2,0,2,2,
130 1,1,1,1,2,1,0,1,2,2,2,1,1,1,1,1,1,1,0,1,1,1,0,0,0,0,0,0,1,1,0,0,
131 2,3,2,3,3,0,0,3,0,1,1,0,1,0,0,0,2,2,1,2,0,0,0,0,0,0,0,0,2,0,1,2,
132 2,2,1,1,1,1,1,2,2,2,1,0,2,0,1,0,1,0,0,1,0,1,0,0,1,0,0,0,0,1,0,0,
133 3,3,3,3,2,2,2,2,2,0,2,1,1,1,1,2,1,2,1,1,0,2,0,1,0,1,0,0,2,0,1,2,
134 1,1,1,1,1,1,1,2,2,1,1,0,2,0,1,0,2,0,0,1,1,1,0,0,2,0,0,0,1,1,0,0,
135 2,3,3,3,3,1,0,0,0,0,0,0,0,0,0,0,2,0,0,1,1,0,0,0,0,0,0,1,2,0,1,2,
136 2,2,2,1,1,2,1,1,2,2,2,1,2,0,1,1,1,1,1,1,0,1,1,1,1,0,0,1,1,1,0,0,
137 2,3,3,3,3,0,2,2,0,2,1,0,0,0,1,1,1,2,0,2,0,0,0,3,0,0,0,0,2,0,2,2,
138 1,1,1,2,1,2,1,1,2,2,2,1,2,0,1,1,1,0,1,1,1,1,0,2,1,0,0,0,1,1,0,0,
139 2,3,3,3,3,0,2,1,0,0,2,0,0,0,0,0,1,2,0,2,0,0,0,0,0,0,0,0,2,0,1,2,
140 1,1,1,2,1,1,1,1,2,2,2,0,1,0,1,1,1,0,0,1,1,1,0,0,1,0,0,0,0,1,0,0,
141 3,3,2,2,3,0,1,0,1,0,0,0,0,0,0,0,1,1,0,3,0,0,0,0,0,0,0,0,1,0,2,2,
142 1,1,1,1,1,2,1,1,2,2,1,2,2,1,0,1,1,1,1,1,0,1,0,0,1,0,0,0,1,1,0,0,
143 3,1,0,1,0,2,2,2,2,3,2,1,1,1,2,3,0,0,1,0,2,1,1,0,1,1,1,1,2,1,1,1,
144 1,2,2,1,2,1,2,2,1,1,0,1,2,1,2,2,1,1,1,0,0,1,1,1,2,1,0,1,0,0,0,0,
145 2,1,0,1,0,3,1,2,2,2,2,1,2,2,1,1,1,0,2,1,2,2,1,1,2,1,1,0,2,1,1,1,
146 1,2,2,2,2,2,2,2,1,2,0,1,1,0,2,1,1,1,1,1,0,0,1,1,1,1,0,1,0,0,0,0,
147 2,1,1,1,1,2,2,2,2,1,2,2,2,1,2,2,1,1,2,1,2,3,2,2,1,1,1,1,0,1,0,0,
148 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
149 2,2,2,3,2,0,1,2,0,1,2,1,1,0,1,0,1,2,1,2,0,0,0,1,1,0,0,0,1,0,0,2,
150 1,1,0,0,1,1,0,1,1,1,1,0,2,0,1,1,1,0,0,1,1,0,0,0,0,1,0,0,0,1,0,0,
151 2,0,0,0,0,1,2,2,2,2,2,2,2,1,2,1,1,1,1,1,1,1,0,1,1,1,1,1,2,1,1,1,
152 1,2,2,2,2,1,1,2,1,2,1,1,1,0,2,1,2,1,1,1,0,2,1,1,1,1,0,1,0,0,0,0,
153 3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,
154 1,1,0,1,0,1,1,1,1,1,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
155 2,2,2,3,2,0,0,0,0,1,0,0,0,0,0,0,1,1,0,2,0,0,0,0,0,0,0,0,1,0,1,2,
156 1,1,1,1,1,1,0,0,2,2,2,2,2,0,1,1,0,1,1,1,1,1,0,0,1,0,0,0,1,1,0,1,
157 2,3,1,2,1,0,1,1,0,2,2,2,0,0,1,0,0,1,1,1,1,0,0,0,0,0,0,0,1,0,1,2,
158 1,1,1,1,2,1,1,1,1,1,1,1,1,0,1,1,0,1,0,1,0,1,0,0,1,0,0,0,0,1,0,0,
159 2,2,2,2,2,0,0,2,0,0,2,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,2,0,2,2,
160 1,1,1,1,1,0,0,1,2,1,1,0,1,0,1,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,
161 1,2,2,2,2,0,0,2,0,1,1,0,0,0,1,0,0,2,0,2,0,0,0,0,0,0,0,0,0,0,1,1,
162 0,0,0,1,1,1,1,1,1,1,1,1,1,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,
163 1,2,2,3,2,0,0,1,0,0,1,0,0,0,0,0,0,1,0,2,0,0,0,1,0,0,0,0,0,0,0,2,
164 1,1,0,0,1,0,0,0,1,1,0,0,1,0,1,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,
165 2,1,2,2,2,1,2,1,2,2,1,1,2,1,1,1,0,1,1,1,1,2,0,1,0,1,1,1,1,0,1,1,
166 1,1,2,1,1,1,1,1,1,0,0,1,2,1,1,1,1,1,1,0,0,1,1,1,0,0,0,0,0,0,0,0,
167 1,0,0,1,3,1,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,
168 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
169 2,2,2,2,1,0,0,1,0,2,0,0,0,0,0,1,1,1,0,1,0,0,0,0,0,0,0,0,2,0,0,1,
170 0,2,0,1,0,0,1,1,2,0,1,0,1,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,
171 1,2,2,2,2,0,1,1,0,2,1,0,1,1,1,0,0,1,0,2,0,1,0,0,0,0,0,0,0,0,0,1,
172 0,1,0,0,1,0,0,0,1,1,0,0,1,0,0,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,
173 2,2,2,2,2,0,0,1,0,0,0,1,0,1,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,1,
174 0,1,0,1,1,1,0,0,1,1,1,0,1,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,
175 2,0,1,0,0,1,2,1,1,1,1,1,1,2,2,1,0,0,1,0,1,0,0,0,0,1,1,1,1,0,0,0,
176 1,1,2,1,1,1,1,0,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
177 2,2,1,2,1,0,0,1,0,0,0,0,0,0,0,0,1,1,0,1,0,0,0,0,0,0,0,0,0,0,0,1,
178 0,0,0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
179 3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
180 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
181 1,0,0,1,2,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,
182 0,1,1,0,1,1,1,0,0,1,0,0,1,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,
183 1,0,1,0,0,1,1,1,1,1,1,1,1,1,1,1,0,0,1,0,2,0,0,2,0,1,0,0,1,0,0,1,
184 1,1,0,0,1,1,0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,
185 0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,
186 1,1,1,1,1,1,1,2,0,0,0,0,0,0,2,1,0,1,1,0,0,1,1,1,0,1,0,0,0,0,0,0,
187 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
188 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
189 1,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,0,1,1,0,1,1,1,1,1,0,1,0,0,
190 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
191 };
192 }
+196
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/CyrillicModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 public class CyrillicModel extends SequenceModel
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final float TYPICAL_POSITIVE_RATIO = 0.976601f;
45
46
47 ////////////////////////////////////////////////////////////////
48 // methods
49 ////////////////////////////////////////////////////////////////
50 public CyrillicModel(short[] charToOrderMap, String name)
51 {
52 super(
53 charToOrderMap,
54 russianLangModel,
55 TYPICAL_POSITIVE_RATIO,
56 false,
57 name
58 );
59 }
60
61
62 ////////////////////////////////////////////////////////////////
63 // constants continued
64 ////////////////////////////////////////////////////////////////
65 private static final byte[] russianLangModel = new byte[] {
66 0,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,1,1,3,3,3,3,1,3,3,3,2,3,2,3,3,
67 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,0,3,2,2,2,2,2,0,0,2,
68 3,3,3,2,3,3,3,3,3,3,3,3,3,3,2,3,3,0,0,3,3,3,3,3,3,3,3,3,2,3,2,0,
69 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
70 3,3,3,2,2,3,3,3,3,3,3,3,3,3,2,3,3,0,0,3,3,3,3,3,3,3,3,2,3,3,1,0,
71 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
72 3,2,3,2,3,3,3,3,3,3,3,3,3,3,3,3,3,0,0,3,3,3,3,3,3,3,3,3,3,3,2,1,
73 0,0,0,0,0,0,0,2,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
74 3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,3,0,0,3,3,3,3,3,3,3,3,3,3,3,2,1,
75 0,0,0,0,0,1,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
76 3,3,3,3,3,3,3,3,2,2,2,3,1,3,3,1,3,3,3,3,2,2,3,0,2,2,2,3,3,2,1,0,
77 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,
78 3,3,3,3,3,3,2,3,3,3,3,3,2,2,3,2,3,3,3,2,1,2,2,0,1,2,2,2,2,2,2,0,
79 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,
80 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,2,3,0,2,2,3,3,2,1,2,0,
81 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,1,0,0,2,0,0,0,0,0,0,0,0,0,
82 3,3,3,3,3,3,2,3,3,1,2,3,2,2,3,2,3,3,3,3,2,2,3,0,3,2,2,3,1,1,1,0,
83 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
84 3,3,3,3,3,3,3,3,2,2,3,3,3,3,3,2,3,3,3,3,2,2,2,0,3,3,3,2,2,2,2,0,
85 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
86 3,3,3,3,3,3,3,3,3,3,2,3,2,3,3,3,3,3,3,2,3,2,2,0,1,3,2,1,2,2,1,0,
87 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,
88 3,3,3,3,3,3,3,3,3,3,3,2,1,1,3,0,1,1,1,1,2,1,1,0,2,2,2,1,2,0,1,0,
89 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
90 3,3,3,3,3,3,2,3,3,2,2,2,2,1,3,2,3,2,3,2,1,2,2,0,1,1,2,1,2,1,2,0,
91 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
92 3,3,3,3,3,3,3,3,3,3,3,3,2,2,3,2,3,3,3,2,2,2,2,0,2,2,2,2,3,1,1,0,
93 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,
94 3,2,3,2,2,3,3,3,3,3,3,3,3,3,1,3,2,0,0,3,3,3,3,2,3,3,3,3,2,3,2,0,
95 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
96 2,3,3,3,3,3,2,2,3,3,0,2,1,0,3,2,3,2,3,0,0,1,2,0,0,1,0,1,2,1,1,0,
97 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
98 3,0,3,0,2,3,3,3,3,2,3,3,3,3,1,2,2,0,0,2,3,2,2,2,3,2,3,2,2,3,0,0,
99 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
100 3,2,3,0,2,3,2,3,0,1,2,3,3,2,0,2,3,0,0,2,3,2,2,0,1,3,1,3,2,2,1,0,
101 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
102 3,1,3,0,2,3,3,3,3,3,3,3,3,2,1,3,2,0,0,2,2,3,3,3,2,3,3,0,2,2,0,0,
103 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
104 3,3,3,3,3,3,2,2,3,3,2,2,2,3,3,0,0,1,1,1,1,1,2,0,0,1,1,1,1,0,1,0,
105 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
106 3,3,3,3,3,3,2,2,3,3,3,3,3,3,3,0,3,2,3,3,2,3,2,0,2,1,0,1,1,0,1,0,
107 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,
108 3,3,3,3,3,3,2,3,3,3,2,2,2,2,3,1,3,2,3,1,1,2,1,0,2,2,2,2,1,3,1,0,
109 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,
110 2,2,3,3,3,3,3,1,2,2,1,3,1,0,3,0,0,3,0,0,0,1,1,0,1,2,1,0,0,0,0,0,
111 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
112 3,2,2,1,1,3,3,3,2,2,1,2,2,3,1,1,2,0,0,2,2,1,3,0,0,2,1,1,2,1,1,0,
113 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
114 3,2,3,3,3,3,1,2,2,2,1,2,1,3,3,1,1,2,1,2,1,2,2,0,2,0,0,1,1,0,1,0,
115 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
116 2,3,3,3,3,3,2,1,3,2,2,3,2,0,3,2,0,3,0,1,0,1,1,0,0,1,1,1,1,0,1,0,
117 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
118 3,3,2,3,3,3,2,2,2,3,3,1,2,1,2,1,0,1,0,1,1,0,1,0,0,2,1,1,1,0,1,0,
119 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,
120 3,1,1,2,1,2,3,3,2,2,1,2,2,3,0,2,1,0,0,2,2,3,2,1,2,2,2,2,2,3,1,0,
121 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
122 3,3,3,3,3,1,1,0,1,1,2,2,1,1,3,0,0,1,3,1,1,1,0,0,0,1,0,1,1,0,0,0,
123 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
124 2,1,3,3,3,2,0,0,0,2,1,0,1,0,2,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
125 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
126 2,0,1,0,0,2,3,2,2,2,1,2,2,2,1,2,1,0,0,1,1,1,0,2,0,1,1,1,0,0,1,1,
127 1,0,0,0,0,0,1,2,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,
128 2,3,3,3,3,0,0,0,0,1,0,0,0,0,3,0,1,2,1,0,0,0,0,0,0,0,1,1,0,0,1,1,
129 1,0,1,0,1,2,0,0,1,1,2,1,0,1,1,1,1,0,1,1,1,1,0,1,0,0,1,0,0,1,1,0,
130 2,2,3,2,2,2,3,1,2,2,2,2,2,2,2,2,1,1,1,1,1,1,1,0,1,0,1,1,1,0,2,1,
131 1,1,1,1,1,1,1,1,2,1,1,1,1,1,1,1,1,1,1,0,1,0,1,1,0,1,1,1,0,1,1,0,
132 3,3,3,2,2,2,2,3,2,2,1,1,2,2,2,2,1,1,3,1,2,1,2,0,0,1,1,0,1,0,2,1,
133 1,1,1,1,1,2,1,0,1,1,1,1,0,1,0,0,1,1,0,0,1,0,1,0,0,1,0,0,0,1,1,0,
134 2,0,0,1,0,3,2,2,2,2,1,2,1,2,1,2,0,0,0,2,1,2,2,1,1,2,2,0,1,1,0,2,
135 1,1,1,1,1,0,1,1,1,2,1,1,1,2,1,0,1,2,1,1,1,1,0,1,1,1,0,0,1,0,0,1,
136 1,3,2,2,2,1,1,1,2,3,0,0,0,0,2,0,2,2,1,0,0,0,0,0,0,1,0,0,0,0,1,1,
137 1,0,1,1,0,1,0,1,1,0,1,1,0,2,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,1,1,0,
138 2,3,2,3,2,1,2,2,2,2,1,0,0,0,2,0,0,1,1,0,0,0,0,0,0,0,1,1,0,0,2,1,
139 1,1,2,1,0,2,0,0,1,0,1,0,0,1,0,0,1,1,0,1,1,0,0,0,0,0,1,0,0,0,0,0,
140 3,0,0,1,0,2,2,2,3,2,2,2,2,2,2,2,0,0,0,2,1,2,1,1,1,2,2,0,0,0,1,2,
141 1,1,1,1,1,0,1,2,1,1,1,1,1,1,1,0,1,1,1,1,1,1,0,1,1,1,1,1,1,0,0,1,
142 2,3,2,3,3,2,0,1,1,1,0,0,1,0,2,0,1,1,3,1,0,0,0,0,0,0,0,1,0,0,2,1,
143 1,1,1,1,1,1,1,0,1,0,1,1,1,1,0,1,1,1,0,0,1,1,0,1,0,0,0,0,0,0,1,0,
144 2,3,3,3,3,1,2,2,2,2,0,1,1,0,2,1,1,1,2,1,0,1,1,0,0,1,0,1,0,0,2,0,
145 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
146 2,3,3,3,2,0,0,1,1,2,2,1,0,0,2,0,1,1,3,0,0,1,0,0,0,0,0,1,0,1,2,1,
147 1,1,2,0,1,1,1,0,1,0,1,1,0,1,0,1,1,1,1,0,1,0,0,0,0,0,0,1,0,1,1,0,
148 1,3,2,3,2,1,0,0,2,2,2,0,1,0,2,0,1,1,1,0,1,0,0,0,3,0,1,1,0,0,2,1,
149 1,1,1,0,1,1,0,0,0,0,1,1,0,1,0,0,2,1,1,0,1,0,0,0,1,0,1,0,0,1,1,0,
150 3,1,2,1,1,2,2,2,2,2,2,1,2,2,1,1,0,0,0,2,2,2,0,0,0,1,2,1,0,1,0,1,
151 2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,2,1,1,1,0,1,0,1,1,0,1,1,1,0,0,1,
152 3,0,0,0,0,2,0,1,1,1,1,1,1,1,0,1,0,0,0,1,1,1,0,1,0,1,1,0,0,1,0,1,
153 1,1,0,0,1,0,0,0,1,0,1,1,0,0,1,0,1,0,1,0,0,0,0,1,0,0,0,1,0,0,0,1,
154 1,3,3,2,2,0,0,0,2,2,0,0,0,1,2,0,1,1,2,0,0,0,0,0,0,0,0,1,0,0,2,1,
155 0,1,1,0,0,1,1,0,0,0,1,1,0,1,1,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,1,0,
156 2,3,2,3,2,0,0,0,0,1,1,0,0,0,2,0,2,0,2,0,0,0,0,0,1,0,0,1,0,0,1,1,
157 1,1,2,0,1,2,1,0,1,1,2,1,1,1,1,1,2,1,1,0,1,0,0,1,1,1,1,1,0,1,1,0,
158 1,3,2,2,2,1,0,0,2,2,1,0,1,2,2,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,1,1,
159 0,0,1,1,0,1,1,0,0,1,1,0,1,1,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,
160 1,0,0,1,0,2,3,1,2,2,2,2,2,2,1,1,0,0,0,1,0,1,0,2,1,1,1,0,0,0,0,1,
161 1,1,0,1,1,0,1,1,1,1,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,
162 2,0,2,0,0,1,0,3,2,1,2,1,2,2,0,1,0,0,0,2,1,0,0,2,1,1,1,1,0,2,0,2,
163 2,1,1,1,1,1,1,1,1,1,1,1,1,2,1,0,1,1,1,1,0,0,0,1,1,1,1,0,1,0,0,1,
164 1,2,2,2,2,1,0,0,1,0,0,0,0,0,2,0,1,1,1,1,0,0,0,0,1,0,1,2,0,0,2,0,
165 1,0,1,1,1,2,1,0,1,0,1,1,0,0,1,0,1,1,1,0,1,0,0,0,1,0,0,1,0,1,1,0,
166 2,1,2,2,2,0,3,0,1,1,0,0,0,0,2,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
167 0,0,0,1,1,1,0,0,1,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,
168 1,2,2,3,2,2,0,0,1,1,2,0,1,2,1,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,0,1,
169 0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,1,1,0,0,1,0,0,0,0,0,0,0,0,1,1,0,
170 2,2,1,1,2,1,2,2,2,2,2,1,2,2,0,1,0,0,0,1,2,2,2,1,2,1,1,1,1,1,2,1,
171 1,1,1,1,1,1,1,1,1,1,0,0,1,1,1,0,1,1,1,0,0,0,0,1,1,1,0,1,1,0,0,1,
172 1,2,2,2,2,0,1,0,2,2,0,0,0,0,2,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,2,0,
173 0,0,1,0,0,1,0,0,0,0,1,0,1,1,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,
174 0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,
175 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
176 1,2,2,2,2,0,0,0,2,2,2,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,1,
177 0,1,1,0,0,1,1,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
178 1,2,2,2,2,0,0,0,0,1,0,0,1,1,2,0,0,0,0,1,0,1,0,0,1,0,0,2,0,0,0,1,
179 0,0,1,0,0,1,0,0,0,1,1,0,0,0,0,0,1,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,
180 1,2,2,2,1,1,2,0,2,1,1,1,1,0,2,2,0,0,0,0,0,0,0,0,0,1,1,0,0,0,1,1,
181 0,0,1,0,1,1,0,0,0,0,1,0,0,0,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,
182 1,0,2,1,2,0,0,0,0,0,1,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,
183 0,0,1,0,1,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,
184 1,0,0,0,0,2,0,1,2,1,0,1,1,1,0,1,0,0,0,1,0,1,0,0,1,0,1,0,0,0,0,1,
185 0,0,0,0,0,1,0,0,1,1,0,0,1,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,
186 2,2,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
187 1,0,0,0,1,0,0,0,1,1,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,
188 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
189 1,1,1,0,1,0,1,0,0,1,1,1,1,0,0,0,1,0,0,0,0,1,0,0,0,1,0,1,0,0,0,0,
190 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
191 1,1,0,1,1,0,1,0,1,0,0,0,0,1,1,0,1,1,0,0,0,0,0,1,0,1,1,0,1,0,0,0,
192 0,1,1,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
193 0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,
194 };
195 }
+194
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/GreekModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 public class GreekModel extends SequenceModel {
40 ////////////////////////////////////////////////////////////////
41 // constants
42 ////////////////////////////////////////////////////////////////
43 public static final float TYPICAL_POSITIVE_RATIO = 0.982851f;
44
45
46 ////////////////////////////////////////////////////////////////
47 // methods
48 ////////////////////////////////////////////////////////////////
49 public GreekModel(short[] charToOrderMap, String name) {
50 super(
51 charToOrderMap,
52 greekLangModel,
53 TYPICAL_POSITIVE_RATIO,
54 false,
55 name
56 );
57 }
58
59
60 ////////////////////////////////////////////////////////////////
61 // constants continued
62 ////////////////////////////////////////////////////////////////
63 private static final byte[] greekLangModel = new byte[] {
64 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
65 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
66 0,0,3,2,2,3,3,3,3,3,3,3,3,1,3,3,3,0,2,2,3,3,0,3,0,3,2,0,3,3,3,0,
67 3,0,0,0,2,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
68 0,3,3,3,3,3,0,3,3,0,3,2,3,3,0,3,2,3,3,3,0,0,3,0,3,0,3,3,2,0,0,0,
69 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,
70 0,2,3,2,2,3,3,3,3,3,3,3,3,0,3,3,3,3,0,2,3,3,0,3,3,3,3,2,3,3,3,0,
71 2,0,0,0,2,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
72 0,2,3,3,2,3,3,3,3,3,3,3,3,3,3,3,3,0,2,1,3,3,3,3,2,3,3,2,3,3,2,0,
73 0,0,0,0,2,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
74 0,3,3,3,3,0,3,3,3,3,3,3,0,3,3,0,3,3,3,3,3,3,3,3,3,3,0,3,2,3,3,0,
75 2,0,1,0,2,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,
76 0,3,3,3,3,3,2,3,0,0,0,0,3,3,0,3,1,3,3,3,0,3,3,0,3,3,3,3,0,0,0,0,
77 2,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
78 0,3,3,3,3,3,0,3,0,3,3,3,3,3,0,3,2,2,2,3,0,2,3,3,3,3,3,2,3,3,0,0,
79 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
80 0,3,3,3,3,3,3,2,2,2,3,3,3,3,0,3,1,3,3,3,3,2,3,3,3,3,3,3,3,2,2,0,
81 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
82 0,3,3,3,3,3,2,0,3,0,0,0,3,3,2,3,3,3,3,3,0,0,3,2,3,0,2,3,0,0,0,0,
83 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
84 0,3,0,3,3,3,3,0,0,3,3,0,2,3,0,3,0,3,3,3,0,0,3,0,3,0,2,2,3,3,0,0,
85 0,0,1,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
86 0,3,3,3,3,3,2,0,3,2,3,3,3,3,0,3,3,3,3,3,0,3,3,2,3,2,3,3,2,0,0,0,
87 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
88 0,3,3,2,3,2,3,3,3,3,3,3,0,2,3,2,3,2,2,2,3,2,3,3,2,3,0,2,2,2,3,0,
89 2,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
90 0,0,3,0,0,0,3,3,3,2,3,3,0,0,3,0,3,0,0,0,3,2,0,3,0,3,0,0,2,0,2,0,
91 0,0,0,0,2,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
92 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
93 0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
94 0,3,3,3,3,0,3,3,3,3,3,3,0,3,3,0,3,0,0,0,3,3,0,3,3,3,0,0,1,2,3,0,
95 3,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
96 0,3,3,3,3,3,2,0,0,3,2,2,3,3,0,3,3,3,3,3,2,1,3,0,3,2,3,3,2,1,0,0,
97 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
98 0,0,3,3,0,2,3,3,3,3,3,3,0,0,3,0,3,0,0,0,3,3,0,3,2,3,0,0,3,3,3,0,
99 3,0,0,0,2,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
100 0,3,3,3,3,0,3,3,3,3,3,3,0,0,3,0,3,0,0,0,3,2,0,3,2,3,0,0,3,2,3,0,
101 2,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
102 0,0,3,1,2,2,3,3,3,3,3,3,0,2,3,0,3,0,0,0,3,3,0,3,0,2,0,0,2,3,1,0,
103 2,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
104 0,3,0,3,3,3,3,0,3,0,3,3,2,3,0,3,3,3,3,3,3,0,3,3,3,0,2,3,0,0,3,0,
105 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
106 0,3,0,3,3,3,0,0,3,0,0,0,3,3,0,3,0,2,3,3,0,0,3,0,3,0,3,3,0,0,0,0,
107 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
108 0,0,3,0,0,0,3,3,3,3,3,3,0,0,3,0,2,0,0,0,3,3,0,3,0,3,0,0,2,0,2,0,
109 0,0,0,0,1,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
110 0,3,3,3,3,3,3,0,3,0,2,0,3,2,0,3,2,3,2,3,0,0,3,2,3,2,3,3,0,0,0,0,
111 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
112 0,0,3,0,0,2,3,3,3,3,3,0,0,0,3,0,2,1,0,0,3,2,2,2,0,3,0,0,2,2,0,0,
113 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
114 0,3,0,3,3,3,2,0,3,0,3,0,3,3,0,2,1,2,3,3,0,0,3,0,3,0,3,3,0,0,0,0,
115 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
116 0,2,3,3,3,0,3,3,3,3,3,3,0,2,3,0,3,0,0,0,2,1,0,2,2,3,0,0,2,2,2,0,
117 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
118 0,0,3,0,0,2,3,3,3,2,3,0,0,1,3,0,2,0,0,0,0,3,0,1,0,2,0,0,1,1,1,0,
119 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
120 0,3,3,3,3,3,1,0,3,0,0,0,3,2,0,3,2,3,3,3,0,0,3,0,3,2,2,2,1,0,0,0,
121 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
122 0,3,0,3,3,3,0,0,3,0,0,0,0,2,0,2,3,3,2,2,2,2,3,0,2,0,2,2,0,0,0,0,
123 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
124 0,3,3,3,3,2,0,0,0,0,0,0,2,3,0,2,0,2,3,2,0,0,3,0,3,0,3,1,0,0,0,0,
125 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
126 0,0,0,0,0,0,3,2,3,3,2,2,3,0,2,0,3,0,0,0,2,0,0,0,0,1,2,0,2,0,2,0,
127 0,2,0,2,0,2,2,0,0,1,0,2,2,2,0,2,2,2,0,2,2,2,0,0,2,0,0,1,0,0,0,0,
128 0,2,0,3,3,2,0,0,0,0,0,0,1,3,0,2,0,2,2,2,0,0,2,0,3,0,0,2,0,0,0,0,
129 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
130 0,3,0,2,3,2,0,2,2,0,2,0,2,2,0,2,0,2,2,2,0,0,0,0,0,0,2,3,0,0,0,2,
131 0,1,2,0,0,0,0,2,2,0,0,0,2,1,0,2,2,0,0,0,0,0,0,1,0,2,0,0,0,0,0,0,
132 0,0,2,1,0,2,3,2,2,3,2,3,2,0,0,3,3,3,0,0,3,2,0,0,0,1,1,0,2,0,2,2,
133 0,2,0,2,0,2,2,0,0,2,0,2,2,2,0,2,2,2,2,0,0,2,0,0,0,2,0,1,0,0,0,0,
134 0,3,0,3,3,2,2,0,3,0,0,0,2,2,0,2,2,2,1,2,0,0,1,2,2,0,0,3,0,0,0,2,
135 0,1,2,0,0,0,1,2,0,0,0,0,0,0,0,2,2,0,1,0,0,2,0,0,0,2,0,0,0,0,0,0,
136 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
137 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
138 0,2,3,3,2,2,0,0,0,2,0,2,3,3,0,2,0,0,0,0,0,0,2,2,2,0,2,2,0,2,0,2,
139 0,2,2,0,0,2,2,2,2,1,0,0,2,2,0,2,0,0,2,0,0,0,0,0,0,2,0,0,0,0,0,0,
140 0,2,0,3,2,3,0,0,0,3,0,0,2,2,0,2,0,2,2,2,0,0,2,0,0,0,0,0,0,0,0,2,
141 0,0,2,2,0,0,2,2,2,0,0,0,0,0,0,2,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,
142 0,0,2,0,0,3,2,0,2,2,2,2,2,0,0,0,2,0,0,0,0,2,0,1,0,0,2,0,1,0,0,0,
143 0,2,2,2,0,2,2,0,1,2,0,2,2,2,0,2,2,2,2,1,2,2,0,0,2,0,0,0,0,0,0,0,
144 0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,
145 0,2,0,2,0,2,2,0,0,0,0,1,2,1,0,0,2,2,0,0,2,0,0,0,0,0,0,0,0,0,0,0,
146 0,0,0,3,2,3,0,0,2,0,0,0,2,2,0,2,0,0,0,1,0,0,2,0,2,0,2,2,0,0,0,0,
147 0,0,2,0,0,0,0,2,2,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,
148 0,2,2,3,2,2,0,0,0,0,0,0,1,3,0,2,0,2,2,0,0,0,1,0,2,0,0,0,0,0,0,0,
149 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
150 0,2,0,2,0,3,2,0,2,0,0,0,0,0,0,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
151 0,0,2,0,0,0,0,1,1,0,0,2,1,2,0,2,2,0,1,0,0,1,0,0,0,2,0,0,0,0,0,0,
152 0,3,0,2,2,2,0,0,2,0,0,0,2,0,0,0,2,3,0,2,0,0,0,0,0,0,2,2,0,0,0,2,
153 0,1,2,0,0,0,1,2,2,1,0,0,0,2,0,0,2,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,
154 0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
155 0,2,1,2,0,2,2,0,2,0,0,2,0,0,0,0,1,2,1,0,2,1,0,0,0,0,0,0,0,0,0,0,
156 0,0,2,0,0,0,3,1,2,2,0,2,0,0,0,0,2,0,0,0,2,0,0,3,0,0,0,0,2,2,2,0,
157 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
158 0,2,1,0,2,0,1,2,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,1,0,0,0,0,0,0,2,
159 0,2,2,0,0,2,2,2,2,2,0,1,2,0,0,0,2,2,0,1,0,2,0,0,2,2,0,0,0,0,0,0,
160 0,0,0,0,1,0,0,0,0,0,0,0,3,0,0,2,0,0,0,0,0,0,0,0,2,0,2,0,0,0,0,2,
161 0,1,2,0,0,0,0,2,2,1,0,1,0,1,0,2,2,2,1,0,0,0,0,0,0,1,0,0,0,0,0,0,
162 0,2,0,1,2,0,0,0,0,0,0,0,0,0,0,2,0,0,2,2,0,0,0,0,1,0,0,0,0,0,0,2,
163 0,2,2,0,0,0,0,2,2,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,2,0,0,2,0,0,0,
164 0,2,2,2,2,0,0,0,3,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,2,0,0,0,0,0,0,1,
165 0,0,2,0,0,0,0,1,2,0,0,0,0,0,0,2,2,1,1,0,0,0,0,0,0,1,0,0,0,0,0,0,
166 0,2,0,2,2,2,0,0,2,0,0,0,0,0,0,0,2,2,2,0,0,0,2,0,0,0,0,0,0,0,0,2,
167 0,0,1,0,0,0,0,2,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,
168 0,3,0,2,0,0,0,0,0,0,0,0,2,0,0,0,0,0,2,0,0,0,0,0,0,0,2,0,0,0,0,2,
169 0,0,2,0,0,0,0,2,2,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
170 0,2,0,2,2,1,0,0,0,0,0,0,2,0,0,2,0,2,2,2,0,0,0,0,0,0,2,0,0,0,0,2,
171 0,0,2,0,0,2,0,2,2,0,0,0,0,2,0,2,0,0,0,0,0,2,0,0,0,2,0,0,0,0,0,0,
172 0,0,3,0,0,0,2,2,0,2,2,0,0,0,0,0,2,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,
173 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
174 0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
175 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,2,0,0,0,0,0,
176 0,2,2,2,2,2,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,1,
177 0,0,0,0,0,0,0,2,1,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
178 0,0,0,0,0,0,0,2,2,0,0,0,0,0,2,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,
179 0,2,0,0,0,2,0,0,0,0,0,1,0,0,0,0,2,2,0,0,0,1,0,0,0,0,0,0,0,0,0,0,
180 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
181 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,2,0,0,0,
182 0,2,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,
183 0,0,1,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
184 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
185 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,2,0,2,0,0,0,
186 0,0,0,0,0,0,0,0,2,1,0,0,0,0,0,0,2,0,0,0,1,2,0,0,0,0,0,0,0,0,0,0,
187 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
188 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
189 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
190 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
191 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
192 };
193 }
+217
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/HebrewModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Simon Montagu <smontagu@smontagu.org>
17 * Portions created by the Initial Developer are Copyright (C) 2005
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shoshannah Forbes <xslf@xslf.com>
22 * Shy Shalom <shooshX@gmail.com>
23 * Kohei TAKETA <k-tak@void.in> (Java port)
24 *
25 * Alternatively, the contents of this file may be used under the terms of
26 * either the GNU General Public License Version 2 or later (the "GPL"), or
27 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
28 * in which case the provisions of the GPL or the LGPL are applicable instead
29 * of those above. If you wish to allow use of your version of this file only
30 * under the terms of either the GPL or the LGPL, and not to allow others to
31 * use your version of this file under the terms of the MPL, indicate your
32 * decision by deleting the provisions above and replace them with the notice
33 * and other provisions required by the GPL or the LGPL. If you do not delete
34 * the provisions above, a recipient may use your version of this file under
35 * the terms of any one of the MPL, the GPL or the LGPL.
36 *
37 * ***** END LICENSE BLOCK ***** */
38
39 package org.mozilla.universalchardet.prober.sequence;
40
41 import org.mozilla.universalchardet.Constants;
42
43
44 public class HebrewModel extends SequenceModel {
45 ////////////////////////////////////////////////////////////////
46 // constants
47 ////////////////////////////////////////////////////////////////
48 public static final float TYPICAL_POSITIVE_RATIO = 0.984004f;
49
50
51 ////////////////////////////////////////////////////////////////
52 // methods
53 ////////////////////////////////////////////////////////////////
54 public HebrewModel() {
55 super(
56 win1255CharToOrderMap,
57 hebrewLangModel,
58 TYPICAL_POSITIVE_RATIO,
59 false,
60 Constants.CHARSET_WINDOWS_1255);
61 }
62
63
64 ////////////////////////////////////////////////////////////////
65 // constants continued
66 ////////////////////////////////////////////////////////////////
67 private static final short[] win1255CharToOrderMap = new short[] {
68 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
69 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
70 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
71 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
72 253, 69, 91, 79, 80, 92, 89, 97, 90, 68,111,112, 82, 73, 95, 85, //40
73 78,121, 86, 71, 67,102,107, 84,114,103,115,253,253,253,253,253, //50
74 253, 50, 74, 60, 61, 42, 76, 70, 64, 53,105, 93, 56, 65, 54, 49, //60
75 66,110, 51, 43, 44, 63, 81, 77, 98, 75,108,253,253,253,253,253, //70
76 124,202,203,204,205, 40, 58,206,207,208,209,210,211,212,213,214,
77 215, 83, 52, 47, 46, 72, 32, 94,216,113,217,109,218,219,220,221,
78 34,116,222,118,100,223,224,117,119,104,125,225,226, 87, 99,227,
79 106,122,123,228, 55,229,230,101,231,232,120,233, 48, 39, 57,234,
80 30, 59, 41, 88, 33, 37, 36, 31, 29, 35,235, 62, 28,236,126,237,
81 238, 38, 45,239,240,241,242,243,127,244,245,246,247,248,249,250,
82 9, 8, 20, 16, 3, 2, 24, 14, 22, 1, 25, 15, 4, 11, 6, 23,
83 12, 19, 13, 26, 18, 27, 21, 17, 7, 10, 5,251,252,128, 96,253,
84 };
85
86 private static final byte[] hebrewLangModel = new byte[] {
87 0,3,3,3,3,3,3,3,3,3,3,2,3,3,3,3,3,3,3,3,3,3,3,2,3,2,1,2,0,1,0,0,
88 3,0,3,1,0,0,1,3,2,0,1,1,2,0,2,2,2,1,1,1,1,2,1,1,1,2,0,0,2,2,0,1,
89 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,2,2,
90 1,2,1,2,1,2,0,0,2,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,
91 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,2,
92 1,2,1,3,1,1,0,0,2,0,0,0,1,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,
93 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,1,0,1,2,2,1,3,
94 1,2,1,1,2,2,0,0,2,2,0,0,0,0,1,0,1,0,0,0,1,0,0,0,0,0,0,1,0,1,1,0,
95 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,3,2,2,2,2,3,2,
96 1,2,1,2,2,2,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,
97 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,3,2,3,2,2,3,2,2,2,1,2,2,2,2,
98 1,2,1,1,2,2,0,1,2,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,
99 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,0,2,2,2,2,2,
100 0,2,0,2,2,2,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,
101 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,0,2,2,2,
102 0,2,1,2,2,2,0,0,2,1,0,0,0,0,1,0,1,0,0,0,0,0,0,2,0,0,0,0,0,0,1,0,
103 3,3,3,3,3,3,3,3,3,3,3,2,3,3,3,3,3,3,3,3,3,3,3,3,3,2,1,2,3,2,2,2,
104 1,2,1,2,2,2,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,1,0,
105 3,3,3,3,3,3,3,3,3,2,3,3,3,2,3,3,3,3,3,3,3,3,3,3,3,3,3,1,0,2,0,2,
106 0,2,1,2,2,2,0,0,1,2,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,2,0,0,1,0,
107 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,2,3,2,2,3,2,1,2,1,1,1,
108 0,1,1,1,1,1,3,0,1,0,0,0,0,2,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
109 3,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1,0,1,1,0,0,1,0,0,1,0,0,0,0,
110 0,0,1,0,0,0,0,0,2,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
111 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,2,2,2,2,2,
112 0,2,0,1,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
113 3,3,3,3,3,3,3,3,3,2,3,3,3,2,1,2,3,3,2,3,3,3,3,2,3,2,1,2,0,2,1,2,
114 0,2,0,2,2,2,0,0,1,2,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,
115 3,3,3,3,3,3,3,3,3,2,3,3,3,1,2,2,3,3,2,3,2,3,2,2,3,1,2,2,0,2,2,2,
116 0,2,1,2,2,2,0,0,1,2,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,1,0,0,1,0,
117 3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,3,3,2,3,3,2,2,2,3,3,3,3,1,3,2,2,2,
118 0,2,0,1,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
119 3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,3,3,3,2,3,2,2,2,1,2,2,0,2,2,2,2,
120 0,2,0,2,2,2,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
121 3,3,3,3,3,3,3,3,3,3,3,2,3,3,3,1,3,2,3,3,2,3,3,2,2,1,2,2,2,2,2,2,
122 0,2,1,2,1,2,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,
123 3,3,3,3,3,3,2,3,2,3,3,2,3,3,3,3,2,3,2,3,3,3,3,3,2,2,2,2,2,2,2,1,
124 0,2,0,1,2,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,
125 3,3,3,3,3,3,3,3,3,2,1,2,3,3,3,3,3,3,3,2,3,2,3,2,1,2,3,0,2,1,2,2,
126 0,2,1,1,2,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,2,0,
127 3,3,3,3,3,3,3,3,3,2,3,3,3,3,2,1,3,1,2,2,2,1,2,3,3,1,2,1,2,2,2,2,
128 0,1,1,1,1,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,2,0,0,0,0,0,0,0,0,
129 3,3,3,3,3,3,3,3,3,3,0,2,3,3,3,1,3,3,3,1,2,2,2,2,1,1,2,2,2,2,2,2,
130 0,2,0,1,1,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,
131 3,3,3,3,3,3,2,3,3,3,2,2,3,3,3,2,1,2,3,2,3,2,2,2,2,1,2,1,1,1,2,2,
132 0,2,1,1,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
133 3,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,0,0,0,1,0,0,0,0,0,
134 1,0,1,0,0,0,0,0,2,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
135 3,3,3,3,3,2,3,3,2,3,1,2,2,2,2,3,2,3,1,1,2,2,1,2,2,1,1,0,2,2,2,2,
136 0,1,0,1,2,2,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,
137 3,0,0,1,1,0,1,0,0,1,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,2,2,0,
138 0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
139 3,0,1,0,1,0,1,1,0,1,1,0,0,0,1,1,0,1,1,1,0,0,0,0,0,0,1,0,0,0,0,0,
140 0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
141 3,0,0,0,1,1,0,1,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,
142 0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,
143 3,2,2,1,2,2,2,2,2,2,2,1,2,2,1,2,2,1,1,1,1,1,1,1,1,2,1,1,0,3,3,3,
144 0,3,0,2,2,2,2,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
145 2,2,2,3,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,1,2,2,1,2,2,2,1,1,1,2,0,1,
146 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
147 2,2,2,2,2,2,2,2,2,2,2,1,2,2,2,2,2,2,2,2,2,2,2,0,2,2,0,0,0,0,0,0,
148 0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
149 2,3,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,1,2,1,0,2,1,0,
150 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
151 3,1,1,1,1,1,1,1,1,1,1,0,0,1,1,1,1,0,1,1,1,1,0,0,0,0,0,0,0,0,0,0,
152 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,
153 0,3,1,1,2,2,2,2,2,1,2,2,2,1,1,2,2,2,2,2,2,2,1,2,2,1,0,1,1,1,1,0,
154 0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
155 3,2,1,1,1,1,2,1,1,2,1,0,1,1,1,1,1,1,1,1,1,1,1,0,1,0,0,0,0,0,0,0,
156 0,0,2,0,0,0,0,0,0,0,0,1,1,0,0,0,0,1,1,0,0,1,1,0,0,0,0,0,0,1,0,0,
157 2,1,1,2,2,2,2,2,2,2,2,2,2,2,1,2,2,2,2,2,1,2,1,2,1,1,1,1,0,0,0,0,
158 0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
159 1,2,1,2,2,2,2,2,2,2,2,2,2,1,2,1,2,1,1,2,1,1,1,2,1,2,1,2,0,1,0,1,
160 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
161 0,3,1,2,2,2,1,2,2,2,2,2,2,2,2,1,2,1,1,1,1,1,1,2,1,2,1,1,0,1,0,1,
162 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
163 2,1,2,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,2,2,
164 0,2,0,1,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
165 3,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,
166 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
167 2,1,1,1,1,1,1,1,0,1,1,0,1,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,
168 0,0,0,0,0,0,0,0,2,0,1,1,1,0,1,0,0,0,1,1,0,1,1,0,0,0,0,0,1,1,0,0,
169 0,1,1,1,2,1,2,2,2,0,2,0,2,0,1,1,2,1,1,1,1,2,1,0,1,1,0,0,0,0,0,0,
170 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
171 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,
172 1,0,1,0,0,0,0,0,1,0,1,2,2,0,1,0,0,1,1,2,2,1,2,0,2,0,0,0,1,2,0,1,
173 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
174 0,0,0,0,0,0,0,0,2,0,2,1,2,0,2,0,0,1,1,1,1,1,1,0,1,0,0,0,1,0,0,1,
175 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
176 0,0,1,0,0,0,0,0,1,0,2,1,1,0,1,0,0,1,1,1,2,2,0,0,1,0,0,0,1,0,0,1,
177 1,1,2,1,0,1,1,1,0,1,0,1,1,1,1,0,0,0,1,0,1,0,0,0,0,0,0,0,0,2,2,1,
178 0,2,0,1,2,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
179 2,1,0,0,1,0,1,1,1,1,0,0,0,0,0,1,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,
180 0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
181 1,1,1,1,1,1,1,1,1,2,1,0,1,1,1,1,1,1,1,1,1,1,1,0,1,0,0,0,0,0,0,0,
182 0,0,0,0,0,0,0,0,0,0,1,1,1,0,0,0,0,1,1,1,0,1,1,0,1,0,0,0,1,1,0,1,
183 2,0,1,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
184 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
185 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
186 0,0,0,0,0,0,0,0,1,0,1,1,1,0,1,0,0,1,1,2,1,1,2,0,1,0,0,0,1,1,0,1,
187 1,0,0,1,0,0,1,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
188 0,0,0,0,0,0,0,0,1,0,1,1,2,0,1,0,0,0,0,2,1,1,2,0,2,0,0,0,1,1,0,1,
189 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
190 0,0,0,0,0,0,0,0,1,0,2,1,1,0,1,0,0,2,2,1,2,1,1,0,1,0,0,0,1,1,0,1,
191 2,0,1,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
192 0,0,0,0,0,0,0,0,0,0,1,2,2,0,0,0,0,0,1,1,0,1,0,0,1,0,0,0,0,1,0,1,
193 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
194 0,0,0,0,0,0,0,0,0,0,1,2,2,0,0,0,0,2,1,1,1,0,2,1,1,0,0,0,2,1,0,1,
195 1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
196 0,0,0,0,0,0,0,0,1,0,1,1,2,0,1,0,0,1,1,0,2,1,1,0,1,0,0,0,1,1,0,1,
197 2,2,1,1,1,0,1,1,0,1,1,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,
198 0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
199 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
200 0,0,0,0,0,0,0,0,1,0,2,1,1,0,1,0,0,1,1,0,1,2,1,0,2,0,0,0,1,1,0,1,
201 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
202 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
203 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
204 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,
205 0,1,0,0,2,0,2,1,1,0,1,0,1,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,
206 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
207 1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
208 0,0,0,0,0,0,0,0,1,0,1,1,2,0,1,0,0,1,1,1,0,1,0,0,1,0,0,0,1,0,0,1,
209 1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
210 1,0,0,0,0,0,0,0,1,0,1,1,0,0,1,0,0,2,1,1,1,1,1,0,1,0,0,0,0,1,0,1,
211 0,1,1,1,2,1,1,1,1,0,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1,0,0,0,0,0,0,0,
212 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
213 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
214 0,0,0,0,0,0,0,0,0,0,1,2,1,0,0,0,0,0,1,1,1,1,1,0,1,0,0,0,1,1,0,0,
215 };
216 }
+73
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/Ibm855Model.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Ibm855Model extends CyrillicModel {
43 ////////////////////////////////////////////////////////////////
44 // methods
45 ////////////////////////////////////////////////////////////////
46 public Ibm855Model() {
47 super(ibm855CharToOrderMap, Constants.CHARSET_IBM855);
48 }
49
50
51 ////////////////////////////////////////////////////////////////
52 // constants
53 ////////////////////////////////////////////////////////////////
54 private static final short[] ibm855CharToOrderMap = new short[] {
55 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
56 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
57 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
58 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
59 253,142,143,144,145,146,147,148,149,150,151,152, 74,153, 75,154, //40
60 155,156,157,158,159,160,161,162,163,164,165,253,253,253,253,253, //50
61 253, 71,172, 66,173, 65,174, 76,175, 64,176,177, 77, 72,178, 69, //60
62 67,179, 78, 73,180,181, 79,182,183,184,185,253,253,253,253,253, //70
63 191,192,193,194, 68,195,196,197,198,199,200,201,202,203,204,205,
64 206,207,208,209,210,211,212,213,214,215,216,217, 27, 59, 54, 70,
65 3, 37, 21, 44, 28, 58, 13, 41, 2, 48, 39, 53, 19, 46,218,219,
66 220,221,222,223,224, 26, 55, 4, 42,225,226,227,228, 23, 60,229,
67 230,231,232,233,234,235, 11, 36,236,237,238,239,240,241,242,243,
68 8, 49, 12, 38, 5, 31, 1, 34, 15,244,245,246,247, 35, 16,248,
69 43, 9, 45, 7, 32, 6, 40, 14, 52, 24, 56, 10, 33, 17, 61,249,
70 250, 18, 62, 20, 51, 25, 57, 30, 47, 29, 63, 22, 50,251,252,255,
71 };
72 }
+73
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/Ibm866Model.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Ibm866Model extends CyrillicModel {
43 ////////////////////////////////////////////////////////////////
44 // methods
45 ////////////////////////////////////////////////////////////////
46 public Ibm866Model() {
47 super(ibm866CharToOrderMap, Constants.CHARSET_IBM866);
48 }
49
50
51 ////////////////////////////////////////////////////////////////
52 // constants
53 ////////////////////////////////////////////////////////////////
54 private static final short[] ibm866CharToOrderMap = new short[] {
55 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
56 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
57 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
58 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
59 253,142,143,144,145,146,147,148,149,150,151,152, 74,153, 75,154, //40
60 155,156,157,158,159,160,161,162,163,164,165,253,253,253,253,253, //50
61 253, 71,172, 66,173, 65,174, 76,175, 64,176,177, 77, 72,178, 69, //60
62 67,179, 78, 73,180,181, 79,182,183,184,185,253,253,253,253,253, //70
63 37, 44, 33, 46, 41, 48, 56, 51, 42, 60, 36, 49, 38, 31, 34, 35,
64 45, 32, 40, 52, 53, 55, 58, 50, 57, 63, 70, 62, 61, 47, 59, 43,
65 3, 21, 10, 19, 13, 2, 24, 20, 4, 23, 11, 8, 12, 5, 1, 15,
66 191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,
67 207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,
68 223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,
69 9, 7, 6, 14, 39, 26, 28, 22, 25, 29, 54, 18, 17, 30, 27, 16,
70 239, 68,240,241,242,243,244,245,246,247,248,249,250,251,252,255,
71 };
72 }
+73
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/Koi8rModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Koi8rModel extends CyrillicModel {
43 ////////////////////////////////////////////////////////////////
44 // methods
45 ////////////////////////////////////////////////////////////////
46 public Koi8rModel() {
47 super(koi8rCharToOrderMap, Constants.CHARSET_KOI8_R);
48 }
49
50
51 ////////////////////////////////////////////////////////////////
52 // constants
53 ////////////////////////////////////////////////////////////////
54 private static final short[] koi8rCharToOrderMap = new short[] {
55 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
56 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
57 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
58 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
59 253,142,143,144,145,146,147,148,149,150,151,152, 74,153, 75,154, //40
60 155,156,157,158,159,160,161,162,163,164,165,253,253,253,253,253, //50
61 253, 71,172, 66,173, 65,174, 76,175, 64,176,177, 77, 72,178, 69, //60
62 67,179, 78, 73,180,181, 79,182,183,184,185,253,253,253,253,253, //70
63 191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206, //80
64 207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222, //90
65 223,224,225, 68,226,227,228,229,230,231,232,233,234,235,236,237, //a0
66 238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253, //b0
67 27, 3, 21, 28, 13, 2, 39, 19, 26, 4, 23, 11, 8, 12, 5, 1, //c0
68 15, 16, 9, 7, 6, 14, 24, 10, 17, 18, 20, 25, 30, 29, 22, 54, //d0
69 59, 37, 44, 58, 41, 48, 53, 46, 55, 42, 60, 36, 49, 38, 31, 34, //e0
70 35, 43, 45, 32, 40, 52, 56, 33, 61, 62, 51, 57, 47, 63, 50, 70, //f0
71 };
72 }
+73
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/Latin5BulgarianModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Latin5BulgarianModel extends BulgarianModel {
43 ////////////////////////////////////////////////////////////////
44 // methods
45 ////////////////////////////////////////////////////////////////
46 public Latin5BulgarianModel() {
47 super(latin5BulgarianCharToOrderMap, Constants.CHARSET_ISO_8859_5);
48 }
49
50
51 ////////////////////////////////////////////////////////////////
52 // constants
53 ////////////////////////////////////////////////////////////////
54 private static final short[] latin5BulgarianCharToOrderMap = new short[] {
55 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
56 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
57 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
58 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
59 253, 77, 90, 99,100, 72,109,107,101, 79,185, 81,102, 76, 94, 82, //40
60 110,186,108, 91, 74,119, 84, 96,111,187,115,253,253,253,253,253, //50
61 253, 65, 69, 70, 66, 63, 68,112,103, 92,194,104, 95, 86, 87, 71, //60
62 116,195, 85, 93, 97,113,196,197,198,199,200,253,253,253,253,253, //70
63 194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209, //80
64 210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225, //90
65 81,226,227,228,229,230,105,231,232,233,234,235,236, 45,237,238, //a0
66 31, 32, 35, 43, 37, 44, 55, 47, 40, 59, 33, 46, 38, 36, 41, 30, //b0
67 39, 28, 34, 51, 48, 49, 53, 50, 54, 57, 61,239, 67,240, 60, 56, //c0
68 1, 18, 9, 20, 11, 3, 23, 15, 2, 26, 12, 10, 14, 6, 4, 13, //d0
69 7, 8, 5, 19, 29, 25, 22, 21, 27, 24, 17, 75, 52,241, 42, 16, //e0
70 62,242,243,244, 58,245, 98,246,247,248,249,250,251, 91,252,253, //f0
71 };
72 }
+73
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/Latin5Model.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Latin5Model extends CyrillicModel {
43 ////////////////////////////////////////////////////////////////
44 // methods
45 ////////////////////////////////////////////////////////////////
46 public Latin5Model() {
47 super(latin5CharToOrderMap, Constants.CHARSET_ISO_8859_5);
48 }
49
50
51 ////////////////////////////////////////////////////////////////
52 // constants
53 ////////////////////////////////////////////////////////////////
54 private static final short[] latin5CharToOrderMap = new short[] {
55 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
56 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
57 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
58 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
59 253,142,143,144,145,146,147,148,149,150,151,152, 74,153, 75,154, //40
60 155,156,157,158,159,160,161,162,163,164,165,253,253,253,253,253, //50
61 253, 71,172, 66,173, 65,174, 76,175, 64,176,177, 77, 72,178, 69, //60
62 67,179, 78, 73,180,181, 79,182,183,184,185,253,253,253,253,253, //70
63 191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,
64 207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,
65 223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,
66 37, 44, 33, 46, 41, 48, 56, 51, 42, 60, 36, 49, 38, 31, 34, 35,
67 45, 32, 40, 52, 53, 55, 58, 50, 57, 63, 70, 62, 61, 47, 59, 43,
68 3, 21, 10, 19, 13, 2, 24, 20, 4, 23, 11, 8, 12, 5, 1, 15,
69 9, 7, 6, 14, 39, 26, 28, 22, 25, 29, 54, 18, 17, 30, 27, 16,
70 239, 68,240,241,242,243,244,245,246,247,248,249,250,251,252,255,
71 };
72 }
+73
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/Latin7Model.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Latin7Model extends GreekModel {
43 ////////////////////////////////////////////////////////////////
44 // methods
45 ////////////////////////////////////////////////////////////////
46 public Latin7Model() {
47 super(latin7CharToOrderMap, Constants.CHARSET_ISO_8859_7);
48 }
49
50
51 ////////////////////////////////////////////////////////////////
52 // constants
53 ////////////////////////////////////////////////////////////////
54 private static final short[] latin7CharToOrderMap = new short[] {
55 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
56 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
57 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
58 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
59 253, 82,100,104, 94, 98,101,116,102,111,187,117, 92, 88,113, 85, //40
60 79,118,105, 83, 67,114,119, 95, 99,109,188,253,253,253,253,253, //50
61 253, 72, 70, 80, 81, 60, 96, 93, 89, 68,120, 97, 77, 86, 69, 55, //60
62 78,115, 65, 66, 58, 76,106,103, 87,107,112,253,253,253,253,253, //70
63 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //80
64 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //90
65 +253,233, 90,253,253,253,253,253,253,253,253,253,253, 74,253,253, //a0
66 253,253,253,253,247,248, 61, 36, 46, 71, 73,253, 54,253,108,123, //b0
67 110, 31, 51, 43, 41, 34, 91, 40, 52, 47, 44, 53, 38, 49, 59, 39, //c0
68 35, 48,250, 37, 33, 45, 56, 50, 84, 57,120,121, 17, 18, 22, 15, //d0
69 124, 1, 29, 20, 21, 3, 32, 13, 25, 5, 11, 16, 10, 6, 30, 4, //e0
70 9, 8, 14, 7, 2, 12, 28, 23, 42, 24, 64, 75, 19, 26, 27,253, //f0
71 };
72 }
+73
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/MacCyrillicModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class MacCyrillicModel extends CyrillicModel {
43 ////////////////////////////////////////////////////////////////
44 // methods
45 ////////////////////////////////////////////////////////////////
46 public MacCyrillicModel() {
47 super(macCyrillicCharToOrderMap, Constants.CHARSET_MACCYRILLIC);
48 }
49
50
51 ////////////////////////////////////////////////////////////////
52 // constants
53 ////////////////////////////////////////////////////////////////
54 private static final short[] macCyrillicCharToOrderMap = new short[] {
55 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
56 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
57 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
58 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
59 253,142,143,144,145,146,147,148,149,150,151,152, 74,153, 75,154, //40
60 155,156,157,158,159,160,161,162,163,164,165,253,253,253,253,253, //50
61 253, 71,172, 66,173, 65,174, 76,175, 64,176,177, 77, 72,178, 69, //60
62 67,179, 78, 73,180,181, 79,182,183,184,185,253,253,253,253,253, //70
63 37, 44, 33, 46, 41, 48, 56, 51, 42, 60, 36, 49, 38, 31, 34, 35,
64 45, 32, 40, 52, 53, 55, 58, 50, 57, 63, 70, 62, 61, 47, 59, 43,
65 191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,
66 207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,
67 223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,
68 239,240,241,242,243,244,245,246,247,248,249,250,251,252, 68, 16,
69 3, 21, 10, 19, 13, 2, 24, 20, 4, 23, 11, 8, 12, 5, 1, 15,
70 9, 7, 6, 14, 39, 26, 28, 22, 25, 29, 54, 18, 17, 30, 27,255,
71 };
72 }
+95
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/SequenceModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei TAKETA <k-taka@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober.sequence;
39
40 public abstract class SequenceModel {
41 ////////////////////////////////////////////////////////////////
42 // fields
43 ////////////////////////////////////////////////////////////////
44 protected short[] charToOrderMap;
45 protected byte[] precedenceMatrix;
46 protected float typicalPositiveRatio;
47 protected boolean keepEnglishLetter;
48 protected String charsetName;
49
50
51 ////////////////////////////////////////////////////////////////
52 // methods
53 ////////////////////////////////////////////////////////////////
54 public SequenceModel(
55 short[] charToOrderMap,
56 byte[] precedenceMatrix,
57 float typicalPositiveRatio,
58 boolean keepEnglishLetter,
59 String charsetName)
60 {
61 super();
62 this.charToOrderMap = charToOrderMap.clone();
63 this.precedenceMatrix = precedenceMatrix.clone();
64 this.typicalPositiveRatio = typicalPositiveRatio;
65 this.keepEnglishLetter = keepEnglishLetter;
66 this.charsetName = charsetName;
67 }
68
69 public short getOrder(byte b)
70 {
71 int c = b & 0xFF;
72 return this.charToOrderMap[c];
73 }
74
75 public byte getPrecedence(int pos)
76 {
77 return this.precedenceMatrix[pos];
78 }
79
80 public float getTypicalPositiveRatio()
81 {
82 return this.typicalPositiveRatio;
83 }
84
85 public boolean getKeepEnglishLetter()
86 {
87 return this.keepEnglishLetter;
88 }
89
90 public String getCharsetName()
91 {
92 return this.charsetName;
93 }
94 }
+239
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/ThaiModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Simon Montagu <smontagu@smontagu.org>
17 * Portions created by the Initial Developer are Copyright (C) 2005
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shoshannah Forbes <xslf@xslf.com>
22 * Shy Shalom <shooshX@gmail.com>
23 * Kohei TAKETA <k-tak@void.in> (Java port)
24 *
25 * Alternatively, the contents of this file may be used under the terms of
26 * either the GNU General Public License Version 2 or later (the "GPL"), or
27 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
28 * in which case the provisions of the GPL or the LGPL are applicable instead
29 * of those above. If you wish to allow use of your version of this file only
30 * under the terms of either the GPL or the LGPL, and not to allow others to
31 * use your version of this file under the terms of the MPL, indicate your
32 * decision by deleting the provisions above and replace them with the notice
33 * and other provisions required by the GPL or the LGPL. If you do not delete
34 * the provisions above, a recipient may use your version of this file under
35 * the terms of any one of the MPL, the GPL or the LGPL.
36 *
37 * ***** END LICENSE BLOCK ***** */
38
39 package org.mozilla.universalchardet.prober.sequence;
40
41 import org.mozilla.universalchardet.Constants;
42
43 public class ThaiModel extends SequenceModel {
44
45 // //////////////////////////////////////////////////////////////
46 // constants
47 // //////////////////////////////////////////////////////////////
48 public static final float TYPICAL_POSITIVE_RATIO = 0.926386f;
49
50 // //////////////////////////////////////////////////////////////
51 // methods
52 // //////////////////////////////////////////////////////////////
53 public ThaiModel() {
54 super(thaiCharToOrderMap, thaiLangModel, TYPICAL_POSITIVE_RATIO, false, Constants.CHARSET_TIS620);
55 }
56
57 // //////////////////////////////////////////////////////////////
58 // constants continued
59 // //////////////////////////////////////////////////////////////
60 private static final short[] thaiCharToOrderMap = new short[] {
61 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 254,
62 255,
63 255,
64 254,
65 255,
66 255, // 00
67 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255,
68 255,
69 255,
70 255,
71 255,
72 255, // 10
73 +253, 253, 253, 253, 253, 253, 253, 253, 253, 253, 253,
74 253,
75 253,
76 253,
77 253,
78 253, // 20
79 252, 252, 252, 252, 252, 252, 252, 252, 252, 252, 253,
80 253,
81 253,
82 253,
83 253,
84 253, // 30
85 253, 182, 106, 107, 100, 183, 184, 185, 101, 94, 186,
86 187,
87 108,
88 109,
89 110,
90 111, // 40
91 188, 189, 190, 89, 95, 112, 113, 191, 192, 193, 194,
92 253,
93 253,
94 253,
95 253,
96 253, // 50
97 253, 64, 72, 73, 114, 74, 115, 116, 102, 81, 201,
98 117,
99 90,
100 103,
101 78,
102 82, // 60
103 96, 202, 91, 79, 84, 104, 105, 97, 98, 92, 203,
104 253,
105 253,
106 253,
107 253,
108 253, // 70
109 209, 210, 211, 212, 213, 88, 214, 215, 216, 217, 218, 219, 220, 118, 221, 222, 223, 224, 99, 85, 83,
110 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 5, 30, 237, 24, 238, 75, 8, 26, 52, 34,
111 51, 119, 47, 58, 57, 49, 53, 55, 43, 20, 19, 44, 14, 48, 3, 17, 25, 39, 62, 31, 54, 45, 9, 16, 2, 61,
112 15, 239, 12, 42, 46, 18, 21, 76, 4, 66, 63, 22, 10, 1, 36, 23, 13, 40, 27, 32, 35, 86, 240, 241, 242,
113 243, 244, 11, 28, 41, 29, 33, 245, 50, 37, 6, 7, 67, 77, 38, 93, 246, 247, 68, 56, 59, 65, 69, 60,
114 70, 80, 71, 87, 248, 249, 250, 251, 252, 253, };
115
116 private static final byte[] thaiLangModel = new byte[] {
117 0, 1, 3, 3, 3, 3, 0, 0, 3, 3, 0, 3, 3, 0, 3, 3, 3, 3, 3, 3, 3, 3, 0, 0, 3, 3, 3, 0, 3, 3, 3, 3, 0, 3,
118 3, 0, 0, 0, 1, 3, 0, 3, 3, 2, 3, 3, 0, 1, 2, 3, 3, 3, 3, 0, 2, 0, 2, 0, 0, 3, 2, 1, 2, 2, 3, 0, 3, 3,
119 2, 3, 0, 0, 3, 3, 0, 3, 3, 0, 3, 3, 3, 3, 3, 3, 3, 3, 3, 0, 3, 2, 3, 0, 2, 2, 2, 3, 0, 2, 3, 0, 0, 0,
120 0, 1, 0, 1, 2, 3, 1, 1, 3, 2, 2, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 3, 3, 3, 2, 3, 3, 3, 3,
121 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 2, 3, 3, 2, 3, 2, 3, 3, 2, 2, 2, 3, 1, 2, 3, 0, 3, 3, 2, 2, 1,
122 2, 3, 3, 1, 2, 0, 1, 3, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 3, 3, 2, 2, 3, 3, 3, 3, 1, 2, 3, 3,
123 3, 3, 3, 2, 2, 2, 2, 3, 3, 2, 2, 3, 3, 2, 2, 3, 2, 3, 2, 2, 3, 3, 1, 2, 3, 1, 2, 2, 3, 3, 1, 0, 2, 1,
124 0, 0, 3, 1, 2, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 3, 3, 3, 3, 3, 3, 2, 2, 3, 3, 3, 3, 2, 3, 2, 2,
125 3, 3, 2, 2, 3, 2, 2, 2, 2, 1, 1, 3, 1, 2, 1, 1, 3, 2, 1, 0, 2, 1, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0,
126 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 2, 3, 2, 3, 3, 2, 2, 3, 2, 3, 3, 2, 3, 1, 1, 2, 3,
127 2, 2, 2, 3, 2, 2, 2, 2, 2, 1, 2, 1, 2, 2, 1, 1, 3, 3, 2, 1, 0, 1, 2, 2, 0, 1, 3, 0, 0, 0, 1, 1, 0, 0,
128 0, 0, 0, 2, 3, 0, 0, 2, 1, 1, 3, 3, 2, 3, 3, 2, 0, 0, 3, 3, 0, 3, 3, 0, 2, 2, 3, 1, 2, 2, 1, 1, 1, 0,
129 2, 2, 2, 0, 2, 2, 1, 1, 0, 2, 1, 0, 2, 0, 0, 2, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0,
130 0, 0, 0, 0, 1, 0, 3, 3, 2, 3, 3, 2, 0, 0, 3, 3, 0, 2, 3, 0, 2, 1, 2, 2, 2, 2, 1, 2, 0, 0, 2, 2, 2, 0,
131 2, 2, 1, 1, 0, 2, 1, 0, 2, 0, 0, 2, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,
132 0, 0, 3, 3, 2, 3, 2, 3, 2, 0, 2, 2, 1, 3, 2, 1, 3, 2, 1, 2, 3, 2, 2, 3, 0, 2, 3, 2, 2, 1, 2, 2, 2, 2,
133 1, 2, 2, 0, 0, 0, 0, 2, 0, 1, 2, 0, 1, 1, 1, 0, 1, 0, 3, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 3, 3,
134 2, 3, 3, 2, 3, 2, 2, 2, 3, 2, 2, 3, 2, 2, 1, 2, 3, 2, 2, 3, 1, 3, 2, 2, 2, 3, 2, 2, 2, 3, 3, 2, 1, 3,
135 0, 1, 1, 1, 0, 2, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 1, 0, 0, 3, 0, 3,
136 3, 3, 3, 3, 0, 0, 3, 0, 2, 2, 3, 3, 3, 3, 3, 0, 0, 0, 1, 1, 3, 0, 0, 0, 0, 2, 0, 0, 1, 0, 0, 0, 0, 0,
137 0, 0, 2, 3, 0, 0, 0, 3, 0, 2, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 3, 3, 3, 3, 0, 0, 2, 3,
138 0, 0, 3, 0, 3, 3, 2, 3, 3, 3, 3, 3, 0, 0, 3, 3, 3, 0, 0, 0, 3, 3, 0, 0, 3, 0, 0, 0, 0, 2, 0, 0, 2, 1,
139 1, 3, 0, 0, 1, 0, 0, 2, 3, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 3, 3, 3, 3, 2, 3, 3, 3, 3, 3, 3, 3, 1, 2,
140 1, 3, 3, 2, 2, 1, 2, 2, 2, 3, 1, 1, 2, 0, 2, 1, 2, 1, 2, 2, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0,
141 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 2, 1, 2, 3, 3, 3, 0, 2, 0, 2, 2, 0, 2, 1, 3, 2,
142 2, 1, 2, 1, 0, 0, 2, 2, 1, 0, 2, 1, 2, 2, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0,
143 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 2, 1, 3, 3, 1, 1, 3, 0, 2, 3, 1, 1, 3, 2, 1, 1, 2, 0,
144 2, 2, 3, 2, 1, 1, 1, 1, 1, 2, 3, 0, 0, 1, 3, 1, 2, 1, 2, 0, 3, 0, 0, 0, 1, 0, 3, 0, 0, 0, 0, 0, 0, 0,
145 0, 0, 0, 0, 0, 1, 0, 0, 3, 3, 1, 1, 3, 2, 3, 3, 3, 1, 3, 2, 1, 3, 2, 1, 3, 2, 2, 2, 2, 1, 3, 3, 1, 2,
146 1, 3, 1, 2, 3, 0, 2, 1, 1, 3, 2, 2, 2, 1, 2, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
147 0, 0, 0, 2, 3, 3, 2, 3, 2, 3, 3, 2, 3, 2, 3, 2, 3, 3, 2, 1, 0, 3, 2, 2, 2, 1, 2, 2, 2, 1, 2, 2, 1, 2,
148 1, 1, 2, 2, 2, 3, 0, 1, 3, 1, 1, 1, 1, 0, 1, 1, 0, 2, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
149 3, 3, 3, 3, 2, 3, 2, 2, 1, 1, 3, 2, 3, 2, 3, 2, 0, 3, 2, 2, 1, 2, 0, 2, 2, 2, 1, 2, 2, 2, 2, 1, 3, 2,
150 1, 2, 2, 1, 0, 2, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 3, 3, 3, 3,
151 3, 2, 3, 1, 2, 3, 3, 2, 2, 3, 0, 1, 1, 2, 0, 3, 3, 2, 2, 3, 0, 1, 1, 3, 0, 0, 0, 0, 3, 1, 0, 3, 3, 0,
152 2, 0, 2, 1, 0, 0, 3, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 2, 3, 2, 3, 3,
153 0, 1, 3, 1, 1, 2, 1, 2, 1, 1, 3, 1, 1, 0, 2, 3, 1, 1, 1, 1, 1, 1, 1, 1, 3, 1, 1, 2, 2, 2, 2, 1, 1, 1,
154 0, 0, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 3, 2, 2, 1, 1, 2, 1, 3, 3, 2, 3, 2,
155 2, 3, 2, 2, 3, 1, 2, 2, 1, 2, 0, 3, 2, 1, 2, 2, 2, 2, 2, 1, 3, 2, 1, 2, 2, 2, 1, 1, 1, 1, 0, 0, 1, 1,
156 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 3, 3, 3, 3, 1, 3, 3, 0, 2, 1, 0, 3,
157 2, 0, 0, 3, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 3, 2, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0,
158 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 2, 2, 2, 3, 0, 0, 1, 3, 0, 3, 2, 0, 3, 2, 2, 3, 3, 3,
159 3, 3, 1, 0, 2, 2, 2, 0, 2, 2, 1, 2, 0, 2, 3, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0,
160 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 3, 0, 2, 3, 1, 3, 3, 2, 3, 3, 0, 3, 3, 0, 3, 2, 2, 3, 2, 3, 3, 3, 0, 0,
161 2, 2, 3, 0, 1, 1, 1, 3, 0, 0, 3, 0, 0, 0, 2, 2, 0, 1, 3, 0, 1, 2, 2, 2, 3, 0, 0, 0, 0, 0, 1, 0, 0, 0,
162 0, 0, 0, 0, 0, 1, 3, 2, 3, 3, 2, 0, 3, 3, 2, 2, 3, 1, 3, 2, 1, 3, 2, 0, 1, 2, 2, 0, 2, 3, 2, 1, 0, 3,
163 0, 0, 0, 0, 3, 0, 0, 2, 3, 1, 3, 0, 0, 3, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
164 0, 0, 3, 1, 3, 2, 2, 2, 1, 2, 0, 1, 3, 1, 1, 3, 1, 3, 0, 0, 2, 1, 1, 1, 1, 2, 1, 1, 1, 0, 2, 1, 0, 1,
165 1, 2, 0, 0, 0, 3, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 3, 1, 0, 0, 0, 1, 0, 3, 3,
166 3, 3, 2, 2, 2, 2, 2, 1, 3, 1, 1, 1, 2, 0, 1, 1, 2, 1, 2, 1, 3, 2, 0, 0, 3, 1, 1, 1, 1, 1, 3, 1, 0, 2,
167 3, 0, 0, 0, 3, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 0,
168 3, 3, 0, 2, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
169 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 1, 3, 0, 0, 1, 2,
170 0, 0, 2, 0, 3, 3, 2, 3, 3, 3, 2, 3, 0, 0, 2, 2, 2, 0, 0, 0, 2, 2, 0, 0, 1, 0, 0, 0, 0, 3, 0, 0, 0, 0,
171 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0,
172 0, 0, 1, 2, 3, 1, 3, 3, 0, 0, 1, 0, 3, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
173 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 1, 2, 3, 1, 2, 3, 1, 0, 3, 0, 2, 2, 1, 0, 2, 1,
174 1, 2, 0, 1, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 3, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0,
175 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 2, 1, 0, 1, 1, 1, 3, 1, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1,
176 0, 3, 1, 0, 1, 3, 1, 1, 1, 1, 1, 1, 0, 2, 0, 1, 3, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,
177 0, 0, 0, 0, 0, 2, 0, 1, 3, 0, 2, 2, 1, 3, 3, 2, 3, 3, 0, 1, 1, 0, 2, 2, 1, 2, 1, 3, 3, 1, 0, 0, 3, 2,
178 0, 0, 0, 0, 2, 1, 0, 1, 0, 0, 0, 0, 1, 2, 0, 1, 1, 3, 1, 1, 2, 2, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0,
179 0, 0, 0, 0, 0, 0, 3, 0, 0, 1, 0, 0, 0, 3, 0, 0, 3, 0, 3, 1, 0, 1, 1, 1, 3, 2, 0, 0, 0, 3, 0, 0, 0, 0,
180 2, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0,
181 3, 3, 1, 3, 2, 1, 3, 3, 1, 2, 2, 0, 1, 2, 1, 0, 1, 2, 0, 0, 0, 0, 0, 3, 0, 0, 0, 3, 0, 0, 0, 0, 3, 0,
182 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 1, 2,
183 0, 3, 3, 3, 2, 2, 0, 1, 1, 0, 1, 3, 0, 0, 0, 2, 2, 0, 0, 0, 0, 3, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0,
184 0, 0, 0, 1, 0, 1, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 2, 3, 1, 2, 0, 0,
185 2, 1, 0, 3, 1, 0, 1, 2, 0, 1, 1, 1, 1, 3, 0, 0, 3, 1, 1, 0, 2, 2, 1, 1, 0, 2, 0, 0, 0, 0, 0, 1, 0, 1,
186 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 3, 1, 2, 0, 0, 2, 2, 0, 1,
187 2, 0, 1, 0, 1, 3, 1, 2, 1, 0, 0, 0, 2, 0, 3, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
188 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 1, 1, 2, 2, 0, 0, 0, 2, 0, 2, 1, 0, 1, 1,
189 0, 1, 1, 1, 2, 1, 0, 0, 1, 1, 1, 0, 2, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0,
190 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 2, 0, 1, 3, 1, 1, 1, 1, 0, 0, 0, 0, 3, 2, 0, 1, 0,
191 0, 0, 1, 2, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
192 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0,
193 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
194 0, 0, 0, 0, 0, 0, 1, 0, 2, 3, 2, 2, 0, 0, 0, 1, 0, 0, 0, 0, 2, 3, 2, 1, 2, 2, 3, 0, 0, 0, 2, 3, 1, 0,
195 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0,
196 0, 0, 3, 3, 2, 2, 0, 1, 0, 0, 0, 0, 2, 0, 2, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 2, 1, 0, 1, 0, 1, 1, 0, 0,
197 0, 1, 0, 2, 0, 0, 1, 0, 3, 0, 1, 0, 0, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3,
198 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 2, 0, 0, 0, 0, 1, 0, 0, 1, 3, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0,
199 0, 0, 3, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 1, 1, 1, 1,
200 2, 3, 0, 0, 2, 1, 1, 1, 1, 1, 0, 2, 1, 1, 0, 0, 0, 2, 1, 0, 1, 2, 1, 1, 0, 1, 2, 1, 0, 3, 0, 0, 0, 0,
201 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 3, 1, 0, 0, 0, 0, 0, 0, 0,
202 3, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 1, 0, 0, 0, 0, 0,
203 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 2, 0, 0, 0, 0, 0, 0, 1, 2, 1, 0, 1,
204 1, 0, 2, 0, 0, 1, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 1, 3, 0, 1, 0, 0,
205 0, 2, 0, 0, 0, 0, 0, 0, 0, 1, 2, 0, 0, 0, 0, 0, 3, 3, 0, 0, 1, 1, 2, 0, 0, 1, 2, 1, 0, 1, 1, 1, 0, 1,
206 1, 0, 0, 2, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 1, 0,
207 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 2, 2, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0,
208 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
209 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 0, 0, 1, 1, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0,
210 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
211 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
212 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
213 1, 1, 0, 1, 2, 0, 1, 2, 0, 0, 1, 1, 0, 2, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 2, 0, 0, 0, 0, 1, 0,
214 0, 1, 0, 1, 1, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0,
215 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 2, 1, 3, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 3, 1, 0, 0, 0, 0, 0,
216 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
217 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0,
218 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 1, 0, 1, 0, 0, 2, 0, 0, 2, 0,
219 0, 1, 1, 2, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
220 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
221 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
222 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
223 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
224 0, 0, 2, 0, 0, 1, 1, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3,
225 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
226 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0,
227 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
228 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
229 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 3, 0, 0, 0, 2, 0,
230 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
231 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
232 0, 0, 0, 1, 0, 0, 0, 0, 2, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
233 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 2, 1, 0, 0,
234 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0,
235 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
236 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
237 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, };
238 }
+73
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/Win1251BulgarianModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Win1251BulgarianModel extends BulgarianModel {
43 ////////////////////////////////////////////////////////////////
44 // methods
45 ////////////////////////////////////////////////////////////////
46 public Win1251BulgarianModel() {
47 super(win1251BulgarianCharToOrderMap, Constants.CHARSET_WINDOWS_1251);
48 }
49
50
51 ////////////////////////////////////////////////////////////////
52 // constants
53 ////////////////////////////////////////////////////////////////
54 private static final short[] win1251BulgarianCharToOrderMap = new short[] {
55 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
56 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
57 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
58 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
59 253, 77, 90, 99,100, 72,109,107,101, 79,185, 81,102, 76, 94, 82, //40
60 110,186,108, 91, 74,119, 84, 96,111,187,115,253,253,253,253,253, //50
61 253, 65, 69, 70, 66, 63, 68,112,103, 92,194,104, 95, 86, 87, 71, //60
62 116,195, 85, 93, 97,113,196,197,198,199,200,253,253,253,253,253, //70
63 206,207,208,209,210,211,212,213,120,214,215,216,217,218,219,220, //80
64 221, 78, 64, 83,121, 98,117,105,222,223,224,225,226,227,228,229, //90
65 88,230,231,232,233,122, 89,106,234,235,236,237,238, 45,239,240, //a0
66 73, 80,118,114,241,242,243,244,245, 62, 58,246,247,248,249,250, //b0
67 31, 32, 35, 43, 37, 44, 55, 47, 40, 59, 33, 46, 38, 36, 41, 30, //c0
68 39, 28, 34, 51, 48, 49, 53, 50, 54, 57, 61,251, 67,252, 60, 56, //d0
69 1, 18, 9, 20, 11, 3, 23, 15, 2, 26, 12, 10, 14, 6, 4, 13, //e0
70 7, 8, 5, 19, 29, 25, 22, 21, 27, 24, 17, 75, 52,253, 42, 16, //f0
71 };
72 }
+73
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/Win1251Model.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Win1251Model extends CyrillicModel {
43 ////////////////////////////////////////////////////////////////
44 // methods
45 ////////////////////////////////////////////////////////////////
46 public Win1251Model() {
47 super(win1251CharToOrderMap, Constants.CHARSET_WINDOWS_1251);
48 }
49
50
51 ////////////////////////////////////////////////////////////////
52 // constants
53 ////////////////////////////////////////////////////////////////
54 private static short[] win1251CharToOrderMap = new short[] {
55 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
56 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
57 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
58 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
59 253,142,143,144,145,146,147,148,149,150,151,152, 74,153, 75,154, //40
60 155,156,157,158,159,160,161,162,163,164,165,253,253,253,253,253, //50
61 253, 71,172, 66,173, 65,174, 76,175, 64,176,177, 77, 72,178, 69, //60
62 67,179, 78, 73,180,181, 79,182,183,184,185,253,253,253,253,253, //70
63 191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,
64 207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,
65 223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,
66 239,240,241,242,243,244,245,246, 68,247,248,249,250,251,252,253,
67 37, 44, 33, 46, 41, 48, 56, 51, 42, 60, 36, 49, 38, 31, 34, 35,
68 45, 32, 40, 52, 53, 55, 58, 50, 57, 63, 70, 62, 61, 47, 59, 43,
69 3, 21, 10, 19, 13, 2, 24, 20, 4, 23, 11, 8, 12, 5, 1, 15,
70 9, 7, 6, 14, 39, 26, 28, 22, 25, 29, 54, 18, 17, 30, 27, 16,
71 };
72 }
+72
-0
src/main/java/org/mozilla/universalchardet/prober/sequence/Win1253Model.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36 package org.mozilla.universalchardet.prober.sequence;
37
38 import org.mozilla.universalchardet.Constants;
39
40
41 public class Win1253Model extends GreekModel {
42 ////////////////////////////////////////////////////////////////
43 // methods
44 ////////////////////////////////////////////////////////////////
45 public Win1253Model() {
46 super(win1253CharToOrderMap, Constants.CHARSET_WINDOWS_1253);
47 }
48
49
50 ////////////////////////////////////////////////////////////////
51 // constants
52 ////////////////////////////////////////////////////////////////
53 private static final short[] win1253CharToOrderMap = new short[] {
54 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
55 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
56 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
57 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
58 253, 82,100,104, 94, 98,101,116,102,111,187,117, 92, 88,113, 85, //40
59 79,118,105, 83, 67,114,119, 95, 99,109,188,253,253,253,253,253, //50
60 253, 72, 70, 80, 81, 60, 96, 93, 89, 68,120, 97, 77, 86, 69, 55, //60
61 78,115, 65, 66, 58, 76,106,103, 87,107,112,253,253,253,253,253, //70
62 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //80
63 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //90
64 253,233, 61,253,253,253,253,253,253,253,253,253,253, 74,253,253, //a0
65 253,253,253,253,247,253,253, 36, 46, 71, 73,253, 54,253,108,123, //b0
66 110, 31, 51, 43, 41, 34, 91, 40, 52, 47, 44, 53, 38, 49, 59, 39, //c0
67 35, 48,250, 37, 33, 45, 56, 50, 84, 57,120,121, 17, 18, 22, 15, //d0
68 124, 1, 29, 20, 21, 3, 32, 13, 25, 5, 11, 16, 10, 6, 30, 4, //e0
69 9, 8, 14, 7, 2, 12, 28, 23, 42, 24, 64, 75, 19, 26, 27,253, //f0
70 };
71 }
+117
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/Big5SMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class Big5SMModel extends SMModel {
47 ////////////////////////////////////////////////////////////////
48 // constants
49 ////////////////////////////////////////////////////////////////
50 public static final int BIG5_CLASS_FACTOR = 5;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public Big5SMModel()
57 {
58 super(
59 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, big5ClassTable),
60 BIG5_CLASS_FACTOR,
61 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, big5StateTable),
62 big5CharLenTable,
63 Constants.CHARSET_BIG5
64 );
65 }
66
67
68 ////////////////////////////////////////////////////////////////
69 // constants continued
70 ////////////////////////////////////////////////////////////////
71 private static int[] big5ClassTable = new int[] {
72 // PkgInt.pack4bits(0,1,1,1,1,1,1,1), // 00 - 07
73 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 00 - 07 //allow 0x00 as legal value
74 PkgInt.pack4bits(1,1,1,1,1,1,0,0), // 08 - 0f
75 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 10 - 17
76 PkgInt.pack4bits(1,1,1,0,1,1,1,1), // 18 - 1f
77 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 20 - 27
78 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 28 - 2f
79 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 30 - 37
80 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 38 - 3f
81 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 40 - 47
82 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 48 - 4f
83 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 50 - 57
84 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 58 - 5f
85 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 60 - 67
86 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 68 - 6f
87 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 70 - 77
88 PkgInt.pack4bits(2,2,2,2,2,2,2,1), // 78 - 7f
89 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 80 - 87
90 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 88 - 8f
91 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 90 - 97
92 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 98 - 9f
93 PkgInt.pack4bits(4,3,3,3,3,3,3,3), // a0 - a7
94 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // a8 - af
95 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // b0 - b7
96 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // b8 - bf
97 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // c0 - c7
98 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // c8 - cf
99 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // d0 - d7
100 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // d8 - df
101 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // e0 - e7
102 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // e8 - ef
103 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // f0 - f7
104 PkgInt.pack4bits(3,3,3,3,3,3,3,0) // f8 - ff
105 };
106
107 private static int[] big5StateTable = new int[] {
108 PkgInt.pack4bits(ERROR,START,START, 3,ERROR,ERROR,ERROR,ERROR),//00-07
109 PkgInt.pack4bits(ERROR,ERROR,ITSME,ITSME,ITSME,ITSME,ITSME,ERROR),//08-0f
110 PkgInt.pack4bits(ERROR,START,START,START,START,START,START,START) //10-17
111 };
112
113 private static int[] big5CharLenTable = new int[] {
114 0, 1, 1, 2, 0
115 };
116 }
+81
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/CodingStateMachine.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 public class CodingStateMachine
40 {
41 protected SMModel model;
42 protected int currentState;
43 protected int currentCharLen;
44 protected int currentBytePos;
45
46 public CodingStateMachine(SMModel model)
47 {
48 this.model = model;
49 this.currentState = SMModel.START;
50 }
51
52 public int nextState(byte c)
53 {
54 int byteCls = this.model.getClass(c);
55 if (this.currentState == SMModel.START) {
56 this.currentBytePos = 0;
57 this.currentCharLen = this.model.getCharLen(byteCls);
58 }
59
60 this.currentState = this.model.getNextState(byteCls, this.currentState);
61 ++this.currentBytePos;
62
63 return this.currentState;
64 }
65
66 public int getCurrentCharLen()
67 {
68 return this.currentCharLen;
69 }
70
71 public void reset()
72 {
73 this.currentState = SMModel.START;
74 }
75
76 public String getCodingStateMachine()
77 {
78 return this.model.getName();
79 }
80 }
+118
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/EUCJPSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class EUCJPSMModel extends SMModel {
47 ////////////////////////////////////////////////////////////////
48 // constants
49 ////////////////////////////////////////////////////////////////
50 public static final int EUCJP_CLASS_FACTOR = 6;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public EUCJPSMModel() {
57 super(
58 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, eucjpClassTable),
59 EUCJP_CLASS_FACTOR,
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, eucjpStateTable),
61 eucjpCharLenTable,
62 Constants.CHARSET_EUC_JP
63 );
64 }
65
66
67 ////////////////////////////////////////////////////////////////
68 // constants continued
69 ////////////////////////////////////////////////////////////////
70 private static int[] eucjpClassTable = new int[] {
71 // PkgInt.pack4bits(5,4,4,4,4,4,4,4), // 00 - 07
72 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 00 - 07
73 PkgInt.pack4bits(4,4,4,4,4,4,5,5), // 08 - 0f
74 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 10 - 17
75 PkgInt.pack4bits(4,4,4,5,4,4,4,4), // 18 - 1f
76 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 20 - 27
77 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 28 - 2f
78 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 30 - 37
79 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 38 - 3f
80 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 40 - 47
81 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 48 - 4f
82 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 50 - 57
83 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 58 - 5f
84 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 60 - 67
85 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 68 - 6f
86 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 70 - 77
87 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 78 - 7f
88 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // 80 - 87
89 PkgInt.pack4bits(5,5,5,5,5,5,1,3), // 88 - 8f
90 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // 90 - 97
91 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // 98 - 9f
92 PkgInt.pack4bits(5,2,2,2,2,2,2,2), // a0 - a7
93 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a8 - af
94 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b0 - b7
95 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b8 - bf
96 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c0 - c7
97 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c8 - cf
98 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d0 - d7
99 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d8 - df
100 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // e0 - e7
101 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // e8 - ef
102 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // f0 - f7
103 PkgInt.pack4bits(0,0,0,0,0,0,0,5) // f8 - ff
104 };
105
106 private static int[] eucjpStateTable = new int[] {
107 PkgInt.pack4bits( 3, 4, 3, 5,START,ERROR,ERROR,ERROR),//00-07
108 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ITSME,ITSME,ITSME),//08-0f
109 PkgInt.pack4bits(ITSME,ITSME,START,ERROR,START,ERROR,ERROR,ERROR),//10-17
110 PkgInt.pack4bits(ERROR,ERROR,START,ERROR,ERROR,ERROR, 3,ERROR),//18-1f
111 PkgInt.pack4bits( 3,ERROR,ERROR,ERROR,START,START,START,START) //20-27
112 };
113
114 private static int[] eucjpCharLenTable = new int[] {
115 2, 2, 2, 3, 1, 0
116 };
117 }
+116
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/EUCKRSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class EUCKRSMModel extends SMModel {
47 ////////////////////////////////////////////////////////////////
48 // constants
49 ////////////////////////////////////////////////////////////////
50 public static final int EUCKR_CLASS_FACTOR = 4;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public EUCKRSMModel() {
57 super(
58 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, euckrClassTable),
59 EUCKR_CLASS_FACTOR,
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, euckrStateTable),
61 euckrCharLenTable,
62 Constants.CHARSET_EUC_KR
63 );
64 }
65
66
67 ////////////////////////////////////////////////////////////////
68 // constants continued
69 ////////////////////////////////////////////////////////////////
70 private static int[] euckrClassTable = new int[] {
71 // PkgInt.pack4bits(0,1,1,1,1,1,1,1), // 00 - 07
72 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 00 - 07
73 PkgInt.pack4bits(1,1,1,1,1,1,0,0), // 08 - 0f
74 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 10 - 17
75 PkgInt.pack4bits(1,1,1,0,1,1,1,1), // 18 - 1f
76 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 20 - 27
77 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 28 - 2f
78 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 30 - 37
79 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 38 - 3f
80 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 40 - 47
81 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 48 - 4f
82 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 50 - 57
83 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 58 - 5f
84 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 60 - 67
85 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 68 - 6f
86 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 70 - 77
87 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 78 - 7f
88 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 80 - 87
89 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 88 - 8f
90 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 90 - 97
91 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 98 - 9f
92 PkgInt.pack4bits(0,2,2,2,2,2,2,2), // a0 - a7
93 PkgInt.pack4bits(2,2,2,2,2,3,3,3), // a8 - af
94 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b0 - b7
95 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b8 - bf
96 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c0 - c7
97 PkgInt.pack4bits(2,3,2,2,2,2,2,2), // c8 - cf
98 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d0 - d7
99 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d8 - df
100 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e0 - e7
101 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e8 - ef
102 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // f0 - f7
103 PkgInt.pack4bits(2,2,2,2,2,2,2,0) // f8 - ff
104 };
105
106 private static int[] euckrStateTable = new int[] {
107 PkgInt.pack4bits(ERROR,START, 3,ERROR,ERROR,ERROR,ERROR,ERROR),//00-07
108 PkgInt.pack4bits(ITSME,ITSME,ITSME,ITSME,ERROR,ERROR,START,START) //08-0f
109 };
110
111 private static int[] euckrCharLenTable = new int[] {
112 0, 1, 2, 0
113 };
114
115 }
+119
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/EUCTWSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class EUCTWSMModel extends SMModel {
47 ////////////////////////////////////////////////////////////////
48 // constants
49 ////////////////////////////////////////////////////////////////
50 public static final int EUCTW_CLASS_FACTOR = 7;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public EUCTWSMModel() {
57 super(
58 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, euctwClassTable),
59 EUCTW_CLASS_FACTOR,
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, euctwStateTable),
61 euctwCharLenTable,
62 Constants.CHARSET_EUC_TW
63 );
64 }
65
66
67 ////////////////////////////////////////////////////////////////
68 // constants continued
69 ////////////////////////////////////////////////////////////////
70 private static int[] euctwClassTable = new int[] {
71 // PkgInt.pack4bits(0,2,2,2,2,2,2,2), // 00 - 07
72 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 00 - 07
73 PkgInt.pack4bits(2,2,2,2,2,2,0,0), // 08 - 0f
74 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 10 - 17
75 PkgInt.pack4bits(2,2,2,0,2,2,2,2), // 18 - 1f
76 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 20 - 27
77 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 28 - 2f
78 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 30 - 37
79 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 38 - 3f
80 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 40 - 47
81 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 48 - 4f
82 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 50 - 57
83 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 58 - 5f
84 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 60 - 67
85 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 68 - 6f
86 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 70 - 77
87 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 78 - 7f
88 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 80 - 87
89 PkgInt.pack4bits(0,0,0,0,0,0,6,0), // 88 - 8f
90 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 90 - 97
91 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 98 - 9f
92 PkgInt.pack4bits(0,3,4,4,4,4,4,4), // a0 - a7
93 PkgInt.pack4bits(5,5,1,1,1,1,1,1), // a8 - af
94 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // b0 - b7
95 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // b8 - bf
96 PkgInt.pack4bits(1,1,3,1,3,3,3,3), // c0 - c7
97 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // c8 - cf
98 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // d0 - d7
99 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // d8 - df
100 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // e0 - e7
101 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // e8 - ef
102 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // f0 - f7
103 PkgInt.pack4bits(3,3,3,3,3,3,3,0) // f8 - ff
104 };
105
106 private static int[] euctwStateTable = new int[] {
107 PkgInt.pack4bits(ERROR,ERROR,START, 3, 3, 3, 4,ERROR),//00-07
108 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ITSME,ITSME),//08-0f
109 PkgInt.pack4bits(ITSME,ITSME,ITSME,ITSME,ITSME,ERROR,START,ERROR),//10-17
110 PkgInt.pack4bits(START,START,START,ERROR,ERROR,ERROR,ERROR,ERROR),//18-1f
111 PkgInt.pack4bits( 5,ERROR,ERROR,ERROR,START,ERROR,START,START),//20-27
112 PkgInt.pack4bits(START,ERROR,START,START,START,START,START,START) //28-2f
113 };
114
115 private static int[] euctwCharLenTable = new int[] {
116 0, 0, 1, 2, 2, 2, 3
117 };
118 }
+118
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/GB18030SMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class GB18030SMModel extends SMModel {
47 ////////////////////////////////////////////////////////////////
48 // constants
49 ////////////////////////////////////////////////////////////////
50 public static final int GB18030_CLASS_FACTOR = 7;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public GB18030SMModel() {
57 super(
58 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, gb18030ClassTable),
59 GB18030_CLASS_FACTOR,
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, gb18030StateTable),
61 gb18030CharLenTable,
62 Constants.CHARSET_GB18030
63 );
64 }
65
66
67 ////////////////////////////////////////////////////////////////
68 // constants continued
69 ////////////////////////////////////////////////////////////////
70 private static int[] gb18030ClassTable = new int[] {
71 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 00 - 07
72 PkgInt.pack4bits(1,1,1,1,1,1,0,0), // 08 - 0f
73 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 10 - 17
74 PkgInt.pack4bits(1,1,1,0,1,1,1,1), // 18 - 1f
75 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 20 - 27
76 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 28 - 2f
77 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // 30 - 37
78 PkgInt.pack4bits(3,3,1,1,1,1,1,1), // 38 - 3f
79 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 40 - 47
80 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 48 - 4f
81 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 50 - 57
82 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 58 - 5f
83 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 60 - 67
84 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 68 - 6f
85 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 70 - 77
86 PkgInt.pack4bits(2,2,2,2,2,2,2,4), // 78 - 7f
87 PkgInt.pack4bits(5,6,6,6,6,6,6,6), // 80 - 87
88 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // 88 - 8f
89 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // 90 - 97
90 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // 98 - 9f
91 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // a0 - a7
92 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // a8 - af
93 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // b0 - b7
94 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // b8 - bf
95 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // c0 - c7
96 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // c8 - cf
97 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // d0 - d7
98 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // d8 - df
99 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // e0 - e7
100 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // e8 - ef
101 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // f0 - f7
102 PkgInt.pack4bits(6,6,6,6,6,6,6,0) // f8 - ff
103 };
104
105 private static int[] gb18030StateTable = new int[] {
106 PkgInt.pack4bits(ERROR,START,START,START,START,START, 3,ERROR),//00-07
107 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ITSME,ITSME),//08-0f
108 PkgInt.pack4bits(ITSME,ITSME,ITSME,ITSME,ITSME,ERROR,ERROR,START),//10-17
109 PkgInt.pack4bits( 4,ERROR,START,START,ERROR,ERROR,ERROR,ERROR),//18-1f
110 PkgInt.pack4bits(ERROR,ERROR, 5,ERROR,ERROR,ERROR,ITSME,ERROR),//20-27
111 PkgInt.pack4bits(ERROR,ERROR,START,START,START,START,START,START) //28-2f
112 };
113
114 private static int[] gb18030CharLenTable = new int[] {
115 0, 1, 1, 1, 1, 1, 2
116 };
117 }
+118
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/HZSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class HZSMModel extends SMModel {
47 ////////////////////////////////////////////////////////////////
48 // constants
49 ////////////////////////////////////////////////////////////////
50 public static final int HZS_CLASS_FACTOR = 6;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public HZSMModel() {
57 super(
58 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, hzsClassTable),
59 HZS_CLASS_FACTOR,
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, hzsStateTable),
61 hzsCharLenTable,
62 Constants.CHARSET_HZ_GB_2312
63 );
64 }
65
66
67 ////////////////////////////////////////////////////////////////
68 // constants continued
69 ////////////////////////////////////////////////////////////////
70 private static int[] hzsClassTable = new int[] {
71 PkgInt.pack4bits(1,0,0,0,0,0,0,0), // 00 - 07
72 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 08 - 0f
73 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 10 - 17
74 PkgInt.pack4bits(0,0,0,1,0,0,0,0), // 18 - 1f
75 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 20 - 27
76 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 28 - 2f
77 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 30 - 37
78 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 38 - 3f
79 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 40 - 47
80 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 48 - 4f
81 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 50 - 57
82 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 58 - 5f
83 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 60 - 67
84 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 68 - 6f
85 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 70 - 77
86 PkgInt.pack4bits(0,0,0,4,0,5,2,0), // 78 - 7f
87 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 80 - 87
88 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 88 - 8f
89 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 90 - 97
90 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 98 - 9f
91 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // a0 - a7
92 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // a8 - af
93 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // b0 - b7
94 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // b8 - bf
95 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // c0 - c7
96 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // c8 - cf
97 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // d0 - d7
98 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // d8 - df
99 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // e0 - e7
100 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // e8 - ef
101 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // f0 - f7
102 PkgInt.pack4bits(1,1,1,1,1,1,1,1) // f8 - ff
103 };
104
105 private static int[] hzsStateTable = new int[] {
106 PkgInt.pack4bits(START,ERROR, 3,START,START,START,ERROR,ERROR),//00-07
107 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ITSME,ITSME,ITSME),//08-0f
108 PkgInt.pack4bits(ITSME,ITSME,ERROR,ERROR,START,START, 4,ERROR),//10-17
109 PkgInt.pack4bits( 5,ERROR, 6,ERROR, 5, 5, 4,ERROR),//18-1f
110 PkgInt.pack4bits( 4,ERROR, 4, 4, 4,ERROR, 4,ERROR),//20-27
111 PkgInt.pack4bits( 4,ITSME,START,START,START,START,START,START) //28-2f
112 };
113
114 private static int[] hzsCharLenTable = new int[] {
115 0, 0, 0, 0, 0, 0
116 };
117 }
+120
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/ISO2022CNSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class ISO2022CNSMModel extends SMModel {
47 ////////////////////////////////////////////////////////////////
48 // constants
49 ////////////////////////////////////////////////////////////////
50 public static final int ISO2022CN_CLASS_FACTOR = 9;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public ISO2022CNSMModel() {
57 super(
58 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, iso2022cnClassTable),
59 ISO2022CN_CLASS_FACTOR,
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, iso2022cnStateTable),
61 iso2022cnCharLenTable,
62 Constants.CHARSET_ISO_2022_CN
63 );
64 }
65
66
67 ////////////////////////////////////////////////////////////////
68 // constants continued
69 ////////////////////////////////////////////////////////////////
70 private static int[] iso2022cnClassTable = new int[] {
71 PkgInt.pack4bits(2,0,0,0,0,0,0,0), // 00 - 07
72 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 08 - 0f
73 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 10 - 17
74 PkgInt.pack4bits(0,0,0,1,0,0,0,0), // 18 - 1f
75 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 20 - 27
76 PkgInt.pack4bits(0,3,0,0,0,0,0,0), // 28 - 2f
77 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 30 - 37
78 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 38 - 3f
79 PkgInt.pack4bits(0,0,0,4,0,0,0,0), // 40 - 47
80 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 48 - 4f
81 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 50 - 57
82 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 58 - 5f
83 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 60 - 67
84 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 68 - 6f
85 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 70 - 77
86 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 78 - 7f
87 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 80 - 87
88 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 88 - 8f
89 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 90 - 97
90 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 98 - 9f
91 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a0 - a7
92 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a8 - af
93 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b0 - b7
94 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b8 - bf
95 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c0 - c7
96 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c8 - cf
97 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d0 - d7
98 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d8 - df
99 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e0 - e7
100 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e8 - ef
101 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // f0 - f7
102 PkgInt.pack4bits(2,2,2,2,2,2,2,2) // f8 - ff
103 };
104
105 private static int[] iso2022cnStateTable = new int[] {
106 PkgInt.pack4bits(START, 3,ERROR,START,START,START,START,START),//00-07
107 PkgInt.pack4bits(START,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//08-0f
108 PkgInt.pack4bits(ERROR,ERROR,ITSME,ITSME,ITSME,ITSME,ITSME,ITSME),//10-17
109 PkgInt.pack4bits(ITSME,ITSME,ITSME,ERROR,ERROR,ERROR, 4,ERROR),//18-1f
110 PkgInt.pack4bits(ERROR,ERROR,ERROR,ITSME,ERROR,ERROR,ERROR,ERROR),//20-27
111 PkgInt.pack4bits( 5, 6,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//28-2f
112 PkgInt.pack4bits(ERROR,ERROR,ERROR,ITSME,ERROR,ERROR,ERROR,ERROR),//30-37
113 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ITSME,ERROR,START) //38-3f
114 };
115
116 private static int[] iso2022cnCharLenTable = new int[] {
117 0, 0, 0, 0, 0, 0, 0, 0, 0
118 };
119 }
+122
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/ISO2022JPSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 * Kazutoshi Satoda
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober.statemachine;
39
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
43 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
44 import org.mozilla.universalchardet.Constants;
45
46
47 public class ISO2022JPSMModel extends SMModel {
48 ////////////////////////////////////////////////////////////////
49 // constants
50 ////////////////////////////////////////////////////////////////
51 public static final int ISO2022JP_CLASS_FACTOR = 10;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public ISO2022JPSMModel() {
58 super(
59 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, iso2022jpClassTable),
60 ISO2022JP_CLASS_FACTOR,
61 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, iso2022jpStateTable),
62 iso2022jpCharLenTable,
63 Constants.CHARSET_ISO_2022_JP
64 );
65 }
66
67
68 ////////////////////////////////////////////////////////////////
69 // constants continued
70 ////////////////////////////////////////////////////////////////
71 private static int[] iso2022jpClassTable = new int[] {
72 PkgInt.pack4bits(2,0,0,0,0,0,0,0), // 00 - 07
73 PkgInt.pack4bits(0,0,0,0,0,0,2,2), // 08 - 0f
74 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 10 - 17
75 PkgInt.pack4bits(0,0,0,1,0,0,0,0), // 18 - 1f
76 PkgInt.pack4bits(0,0,0,0,7,0,0,0), // 20 - 27
77 PkgInt.pack4bits(3,0,0,0,0,0,0,0), // 28 - 2f
78 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 30 - 37
79 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 38 - 3f
80 PkgInt.pack4bits(6,0,4,0,8,0,0,0), // 40 - 47
81 PkgInt.pack4bits(0,9,5,0,0,0,0,0), // 48 - 4f
82 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 50 - 57
83 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 58 - 5f
84 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 60 - 67
85 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 68 - 6f
86 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 70 - 77
87 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 78 - 7f
88 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 80 - 87
89 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 88 - 8f
90 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 90 - 97
91 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 98 - 9f
92 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a0 - a7
93 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a8 - af
94 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b0 - b7
95 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b8 - bf
96 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c0 - c7
97 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c8 - cf
98 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d0 - d7
99 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d8 - df
100 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e0 - e7
101 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e8 - ef
102 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // f0 - f7
103 PkgInt.pack4bits(2,2,2,2,2,2,2,2) // f8 - ff
104 };
105
106 private static int[] iso2022jpStateTable = new int[] {
107 PkgInt.pack4bits(START, 3,ERROR,START,START,START,START,START),//00-07
108 PkgInt.pack4bits(START,START,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//08-0f
109 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ITSME,ITSME,ITSME),//10-17
110 PkgInt.pack4bits(ITSME,ITSME,ITSME,ITSME,ITSME,ITSME,ERROR,ERROR),//18-1f
111 PkgInt.pack4bits(ERROR, 5,ERROR,ERROR,ERROR, 4,ERROR,ERROR),//20-27
112 PkgInt.pack4bits(ERROR,ERROR,ERROR, 6,ITSME,ERROR,ITSME,ERROR),//28-2f
113 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ITSME,ITSME),//30-37
114 PkgInt.pack4bits(ERROR,ERROR,ERROR,ITSME,ERROR,ERROR,ERROR,ERROR),//38-3f
115 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ERROR,START,START) //40-47
116 };
117
118 private static int[] iso2022jpCharLenTable = new int[] {
119 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
120 };
121 }
+116
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/ISO2022KRSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45 public class ISO2022KRSMModel extends SMModel {
46 ////////////////////////////////////////////////////////////////
47 // constants
48 ////////////////////////////////////////////////////////////////
49 public static final int ISO2022KR_CLASS_FACTOR = 6;
50
51
52 ////////////////////////////////////////////////////////////////
53 // methods
54 ////////////////////////////////////////////////////////////////
55 public ISO2022KRSMModel() {
56 super(
57 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, iso2022krClassTable),
58 ISO2022KR_CLASS_FACTOR,
59 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, iso2022krStateTable),
60 iso2022krCharLenTable,
61 Constants.CHARSET_ISO_2022_KR
62 );
63 }
64
65
66 ////////////////////////////////////////////////////////////////
67 // constants continued
68 ////////////////////////////////////////////////////////////////
69 private static int[] iso2022krClassTable = new int[] {
70 PkgInt.pack4bits(2,0,0,0,0,0,0,0), // 00 - 07
71 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 08 - 0f
72 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 10 - 17
73 PkgInt.pack4bits(0,0,0,1,0,0,0,0), // 18 - 1f
74 PkgInt.pack4bits(0,0,0,0,3,0,0,0), // 20 - 27
75 PkgInt.pack4bits(0,4,0,0,0,0,0,0), // 28 - 2f
76 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 30 - 37
77 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 38 - 3f
78 PkgInt.pack4bits(0,0,0,5,0,0,0,0), // 40 - 47
79 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 48 - 4f
80 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 50 - 57
81 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 58 - 5f
82 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 60 - 67
83 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 68 - 6f
84 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 70 - 77
85 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 78 - 7f
86 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 80 - 87
87 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 88 - 8f
88 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 90 - 97
89 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 98 - 9f
90 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a0 - a7
91 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a8 - af
92 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b0 - b7
93 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b8 - bf
94 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c0 - c7
95 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c8 - cf
96 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d0 - d7
97 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d8 - df
98 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e0 - e7
99 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e8 - ef
100 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // f0 - f7
101 PkgInt.pack4bits(2,2,2,2,2,2,2,2) // f8 - ff
102 };
103
104 private static int[] iso2022krStateTable = new int[] {
105 PkgInt.pack4bits(START, 3,ERROR,START,START,START,ERROR,ERROR),//00-07
106 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ITSME,ITSME,ITSME),//08-0f
107 PkgInt.pack4bits(ITSME,ITSME,ERROR,ERROR,ERROR, 4,ERROR,ERROR),//10-17
108 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR, 5,ERROR,ERROR,ERROR),//18-1f
109 PkgInt.pack4bits(ERROR,ERROR,ERROR,ITSME,START,START,START,START) //20-27
110 };
111
112 private static int[] iso2022krCharLenTable = new int[] {
113 0, 0, 0, 0, 0, 0
114 };
115 }
+117
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/PkgInt.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 public class PkgInt {
40 ////////////////////////////////////////////////////////////////
41 // constants
42 ////////////////////////////////////////////////////////////////
43 public static final int INDEX_SHIFT_4BITS = 3;
44 public static final int INDEX_SHIFT_8BITS = 2;
45 public static final int INDEX_SHIFT_16BITS = 1;
46
47 public static final int SHIFT_MASK_4BITS = 7;
48 public static final int SHIFT_MASK_8BITS = 3;
49 public static final int SHIFT_MASK_16BITS = 1;
50
51 public static final int BIT_SHIFT_4BITS = 2;
52 public static final int BIT_SHIFT_8BITS = 3;
53 public static final int BIT_SHIFT_16BITS = 4;
54
55 public static final int UNIT_MASK_4BITS = 0x0000000F;
56 public static final int UNIT_MASK_8BITS = 0x000000FF;
57 public static final int UNIT_MASK_16BITS = 0x0000FFFF;
58
59
60 ////////////////////////////////////////////////////////////////
61 // fields
62 ////////////////////////////////////////////////////////////////
63 private int indexShift;
64 private int shiftMask;
65 private int bitShift;
66 private int unitMask;
67 private int[] data;
68
69
70 ////////////////////////////////////////////////////////////////
71 // methods
72 ////////////////////////////////////////////////////////////////
73 public PkgInt(
74 int indexShift,
75 int shiftMask,
76 int bitShift,
77 int unitMask,
78 int[] data)
79 {
80 this.indexShift = indexShift;
81 this.shiftMask = shiftMask;
82 this.bitShift = bitShift;
83 this.unitMask = unitMask;
84 this.data = data.clone();
85 }
86
87 public static int pack16bits(int a, int b)
88 {
89 return ((b << 16) | a);
90 }
91
92 public static int pack8bits(int a, int b, int c, int d)
93 {
94 return pack16bits(
95 (b << 8) | a,
96 (d << 8) | c
97 );
98 }
99
100 public static int pack4bits(int a, int b, int c, int d, int e, int f, int g, int h)
101 {
102 return pack8bits(
103 (b << 4) | a,
104 (d << 4) | c,
105 (f << 4) | e,
106 (h << 4) | g
107 );
108 }
109
110 public int unpack(int i)
111 {
112 return (
113 (this.data[i>>this.indexShift] >> ((i&this.shiftMask)<<this.bitShift)) & this.unitMask
114 );
115 }
116 }
+118
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/SJISSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class SJISSMModel extends SMModel {
47 ////////////////////////////////////////////////////////////////
48 // constants
49 ////////////////////////////////////////////////////////////////
50 public static final int SJIS_CLASS_FACTOR = 6;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public SJISSMModel() {
57 super(
58 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, sjisClassTable),
59 SJIS_CLASS_FACTOR,
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, sjisStateTable),
61 sjisCharLenTable,
62 Constants.CHARSET_SHIFT_JIS
63 );
64 }
65
66
67 ////////////////////////////////////////////////////////////////
68 // constants continued
69 ////////////////////////////////////////////////////////////////
70 private static int[] sjisClassTable = new int[] {
71 // PkgInt.pack4bits(0,1,1,1,1,1,1,1), // 00 - 07
72 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 00 - 07
73 PkgInt.pack4bits(1,1,1,1,1,1,0,0), // 08 - 0f
74 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 10 - 17
75 PkgInt.pack4bits(1,1,1,0,1,1,1,1), // 18 - 1f
76 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 20 - 27
77 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 28 - 2f
78 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 30 - 37
79 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 38 - 3f
80 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 40 - 47
81 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 48 - 4f
82 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 50 - 57
83 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 58 - 5f
84 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 60 - 67
85 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 68 - 6f
86 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 70 - 77
87 PkgInt.pack4bits(2,2,2,2,2,2,2,1), // 78 - 7f
88 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // 80 - 87
89 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // 88 - 8f
90 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // 90 - 97
91 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // 98 - 9f
92 // 0xa0 is illegal in sjis encoding, but some pages does
93 // contain such byte. We need to be more error forgiven.
94 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a0 - a7
95 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a8 - af
96 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b0 - b7
97 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b8 - bf
98 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c0 - c7
99 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c8 - cf
100 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d0 - d7
101 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d8 - df
102 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // e0 - e7
103 PkgInt.pack4bits(3,3,3,3,3,4,4,4), // e8 - ef
104 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // f0 - f7
105 PkgInt.pack4bits(4,4,4,4,4,0,0,0) // f8 - ff
106 };
107
108 private static int[] sjisStateTable = new int[] {
109 PkgInt.pack4bits(ERROR,START,START, 3,ERROR,ERROR,ERROR,ERROR),//00-07
110 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ITSME,ITSME,ITSME),//08-0f
111 PkgInt.pack4bits(ITSME,ITSME,ERROR,ERROR,START,START,START,START) //10-17
112 };
113
114 private static int[] sjisCharLenTable = new int[] {
115 0, 1, 1, 2, 0, 0
116 };
117 }
+93
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/SMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 public abstract class SMModel {
40 ////////////////////////////////////////////////////////////////
41 // constants
42 ////////////////////////////////////////////////////////////////
43 public static final int START = 0;
44 public static final int ERROR = 1;
45 public static final int ITSME = 2;
46
47
48 ////////////////////////////////////////////////////////////////
49 // fields
50 ////////////////////////////////////////////////////////////////
51 protected PkgInt classTable;
52 protected int classFactor;
53 protected PkgInt stateTable;
54 protected int[] charLenTable;
55 protected String name;
56
57
58 ////////////////////////////////////////////////////////////////
59 // methods
60 ////////////////////////////////////////////////////////////////
61 public SMModel(
62 PkgInt classTable,
63 int classFactor,
64 PkgInt stateTable,
65 int[] charLenTable,
66 String name)
67 {
68 super();
69 this.classTable = classTable;
70 this.classFactor = classFactor;
71 this.stateTable = stateTable;
72 this.charLenTable = charLenTable.clone();
73 this.name = name;
74 }
75
76 public int getClass(byte b) {
77 int c = b & 0xFF;
78 return this.classTable.unpack(c);
79 }
80
81 public int getNextState(int cls, int currentState) {
82 return this.stateTable.unpack(currentState * this.classFactor + cls);
83 }
84
85 public int getCharLen(int cls) {
86 return this.charLenTable[cls];
87 }
88
89 public String getName() {
90 return this.name;
91 }
92 }
+119
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/UCS2BESMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class UCS2BESMModel extends SMModel {
47 ////////////////////////////////////////////////////////////////
48 // constants
49 ////////////////////////////////////////////////////////////////
50 public static final int UCS2BE_CLASS_FACTOR = 6;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public UCS2BESMModel() {
57 super(
58 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, ucs2beClassTable),
59 UCS2BE_CLASS_FACTOR,
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, ucs2beStateTable),
61 ucs2beCharLenTable,
62 Constants.CHARSET_UTF_16BE
63 );
64 }
65
66
67 ////////////////////////////////////////////////////////////////
68 // constants continued
69 ////////////////////////////////////////////////////////////////
70 private static int[] ucs2beClassTable = new int[] {
71 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 00 - 07
72 PkgInt.pack4bits(0,0,1,0,0,2,0,0), // 08 - 0f
73 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 10 - 17
74 PkgInt.pack4bits(0,0,0,3,0,0,0,0), // 18 - 1f
75 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 20 - 27
76 PkgInt.pack4bits(0,3,3,3,3,3,0,0), // 28 - 2f
77 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 30 - 37
78 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 38 - 3f
79 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 40 - 47
80 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 48 - 4f
81 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 50 - 57
82 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 58 - 5f
83 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 60 - 67
84 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 68 - 6f
85 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 70 - 77
86 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 78 - 7f
87 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 80 - 87
88 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 88 - 8f
89 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 90 - 97
90 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 98 - 9f
91 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // a0 - a7
92 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // a8 - af
93 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // b0 - b7
94 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // b8 - bf
95 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // c0 - c7
96 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // c8 - cf
97 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // d0 - d7
98 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // d8 - df
99 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // e0 - e7
100 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // e8 - ef
101 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // f0 - f7
102 PkgInt.pack4bits(0,0,0,0,0,0,4,5) // f8 - ff
103 };
104
105 private static int[] ucs2beStateTable = new int[] {
106 PkgInt.pack4bits( 5, 7, 7,ERROR, 4, 3,ERROR,ERROR),//00-07
107 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ITSME,ITSME,ITSME),//08-0f
108 PkgInt.pack4bits(ITSME,ITSME, 6, 6, 6, 6,ERROR,ERROR),//10-17
109 PkgInt.pack4bits( 6, 6, 6, 6, 6,ITSME, 6, 6),//18-1f
110 PkgInt.pack4bits( 6, 6, 6, 6, 5, 7, 7,ERROR),//20-27
111 PkgInt.pack4bits( 5, 8, 6, 6,ERROR, 6, 6, 6),//28-2f
112 PkgInt.pack4bits( 6, 6, 6, 6,ERROR,ERROR,START,START) //30-37
113 };
114
115 private static int[] ucs2beCharLenTable = new int[] {
116 2, 2, 2, 0, 2, 2
117 };
118 }
+140
-0
src/main/java/org/mozilla/universalchardet/prober/statemachine/UTF8SMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 * Aaron Madlon-Kay (emoji fix)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober.statemachine;
39
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
43 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
44 import org.mozilla.universalchardet.Constants;
45
46 public class UTF8SMModel extends SMModel {
47 ////////////////////////////////////////////////////////////////
48 // constants
49 ////////////////////////////////////////////////////////////////
50 public static final int UTF8_CLASS_FACTOR = 16;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public UTF8SMModel() {
57 super(
58 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, utf8ClassTable),
59 UTF8_CLASS_FACTOR,
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, utf8StateTable),
61 utf8CharLenTable,
62 Constants.CHARSET_UTF_8
63 );
64 }
65
66
67 ////////////////////////////////////////////////////////////////
68 // constants continued
69 ////////////////////////////////////////////////////////////////
70 private static int[] utf8ClassTable = new int[] {
71 // PkgInt.pack4bits(0,1,1,1,1,1,1,1), // 00 - 07
72 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 00 - 07 //allow 0x00 as a legal value
73 PkgInt.pack4bits(1,1,1,1,1,1,0,0), // 08 - 0f
74 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 10 - 17
75 PkgInt.pack4bits(1,1,1,0,1,1,1,1), // 18 - 1f
76 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 20 - 27
77 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 28 - 2f
78 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 30 - 37
79 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 38 - 3f
80 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 40 - 47
81 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 48 - 4f
82 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 50 - 57
83 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 58 - 5f
84 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 60 - 67
85 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 68 - 6f
86 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 70 - 77
87 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 78 - 7f
88 PkgInt.pack4bits(2,2,2,2,3,3,3,3), // 80 - 87
89 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 88 - 8f
90 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 90 - 97
91 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 98 - 9f
92 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // a0 - a7
93 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // a8 - af
94 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // b0 - b7
95 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // b8 - bf
96 PkgInt.pack4bits(0,0,6,6,6,6,6,6), // c0 - c7
97 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // c8 - cf
98 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // d0 - d7
99 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // d8 - df
100 PkgInt.pack4bits(7,8,8,8,8,8,8,8), // e0 - e7
101 PkgInt.pack4bits(8,8,8,8,8,9,8,8), // e8 - ef
102 PkgInt.pack4bits(10,11,11,11,11,11,11,11), // f0 - f7
103 PkgInt.pack4bits(12,13,13,13,14,15,0,0) // f8 - ff
104 };
105
106 private static int[] utf8StateTable = new int[] {
107 PkgInt.pack4bits(ERROR,START,ERROR,ERROR,ERROR,ERROR, 12, 10),//00-07
108 PkgInt.pack4bits( 9, 11, 8, 7, 6, 5, 4, 3),//08-0f
109 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//10-17
110 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//18-1f
111 PkgInt.pack4bits(ITSME,ITSME,ITSME,ITSME,ITSME,ITSME,ITSME,ITSME),//20-27
112 PkgInt.pack4bits(ITSME,ITSME,ITSME,ITSME,ITSME,ITSME,ITSME,ITSME),//28-2f
113 PkgInt.pack4bits(ERROR,ERROR, 5, 5, 5, 5,ERROR,ERROR),//30-37
114 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//38-3f
115 PkgInt.pack4bits(ERROR,ERROR,ERROR, 5, 5, 5,ERROR,ERROR),//40-47
116 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//48-4f
117 PkgInt.pack4bits(ERROR,ERROR, 7, 7, 7, 7,ERROR,ERROR),//50-57
118 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//58-5f
119 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR, 7, 7,ERROR,ERROR),//60-67
120 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//68-6f
121 PkgInt.pack4bits(ERROR,ERROR, 9, 9, 9, 9,ERROR,ERROR),//70-77
122 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//78-7f
123 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR, 9, 9,ERROR,ERROR),//80-87 // Allow emoji
124 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//88-8f
125 PkgInt.pack4bits(ERROR,ERROR, 12, 12, 12, 12,ERROR,ERROR),//90-97
126 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//98-9f
127 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR, 12,ERROR,ERROR),//a0-a7
128 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//a8-af
129 PkgInt.pack4bits(ERROR,ERROR, 12, 12, 12,ERROR,ERROR,ERROR),//b0-b7
130 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//b8-bf
131 PkgInt.pack4bits(ERROR,ERROR,START,START,START,START,ERROR,ERROR),//c0-c7
132 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR) //c8-cf
133 };
134
135 private static int[] utf8CharLenTable = new int[] {
136 0, 1, 0, 0, 0, 0, 2, 3,
137 3, 3, 4, 4, 5, 5, 6, 6
138 };
139 }
+0
-43
src/org/mozilla/universalchardet/CharsetListener.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet;
38
39 public interface CharsetListener
40 {
41 public void report(String charset);
42 }
+0
-35
src/org/mozilla/universalchardet/Constants.java less more
0 package org.mozilla.universalchardet;
1
2 public final class Constants
3 {
4 public static final String CHARSET_ISO_2022_JP = "ISO-2022-JP".intern();
5 public static final String CHARSET_ISO_2022_CN = "ISO-2022-CN".intern();
6 public static final String CHARSET_ISO_2022_KR = "ISO-2022-KR".intern();
7 public static final String CHARSET_ISO_8859_5 = "ISO-8859-5".intern();
8 public static final String CHARSET_ISO_8859_7 = "ISO-8859-7".intern();
9 public static final String CHARSET_ISO_8859_8 = "ISO-8859-8".intern();
10 public static final String CHARSET_BIG5 = "BIG5".intern();
11 public static final String CHARSET_GB18030 = "GB18030".intern();
12 public static final String CHARSET_EUC_JP = "EUC-JP".intern();
13 public static final String CHARSET_EUC_KR = "EUC-KR".intern();
14 public static final String CHARSET_EUC_TW = "EUC-TW".intern();
15 public static final String CHARSET_SHIFT_JIS = "SHIFT_JIS".intern();
16 public static final String CHARSET_IBM855 = "IBM855".intern();
17 public static final String CHARSET_IBM866 = "IBM866".intern();
18 public static final String CHARSET_KOI8_R = "KOI8-R".intern();
19 public static final String CHARSET_MACCYRILLIC = "MACCYRILLIC".intern();
20 public static final String CHARSET_WINDOWS_1251 = "WINDOWS-1251".intern();
21 public static final String CHARSET_WINDOWS_1252 = "WINDOWS-1252".intern();
22 public static final String CHARSET_WINDOWS_1253 = "WINDOWS-1253".intern();
23 public static final String CHARSET_WINDOWS_1255 = "WINDOWS-1255".intern();
24 public static final String CHARSET_UTF_8 = "UTF-8".intern();
25 public static final String CHARSET_UTF_16BE = "UTF-16BE".intern();
26 public static final String CHARSET_UTF_16LE = "UTF-16LE".intern();
27 public static final String CHARSET_UTF_32BE = "UTF-32BE".intern();
28 public static final String CHARSET_UTF_32LE = "UTF-32LE".intern();
29
30 // WARNING: Listed below are charsets which Java does not support.
31 public static final String CHARSET_HZ_GB_2312 = "HZ-GB-2312".intern(); // Simplified Chinese
32 public static final String CHARSET_X_ISO_10646_UCS_4_3412 = "X-ISO-10646-UCS-4-3412".intern(); // Malformed UTF-32
33 public static final String CHARSET_X_ISO_10646_UCS_4_2143 = "X-ISO-10646-UCS-4-2143".intern(); // Malformed UTF-32
34 }
+0
-324
src/org/mozilla/universalchardet/UniversalDetector.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei TAKETA <k-tak@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet;
39
40 import org.mozilla.universalchardet.prober.CharsetProber;
41 import org.mozilla.universalchardet.prober.MBCSGroupProber;
42 import org.mozilla.universalchardet.prober.SBCSGroupProber;
43 import org.mozilla.universalchardet.prober.EscCharsetProber;
44 import org.mozilla.universalchardet.prober.Latin1Prober;
45 import org.mozilla.universalchardet.Constants;
46
47 public class UniversalDetector
48 {
49 ////////////////////////////////////////////////////////////////
50 // constants
51 ////////////////////////////////////////////////////////////////
52 public static final float SHORTCUT_THRESHOLD = 0.95f;
53 public static final float MINIMUM_THRESHOLD = 0.20f;
54
55
56 ////////////////////////////////////////////////////////////////
57 // inner types
58 ////////////////////////////////////////////////////////////////
59 public enum InputState
60 {
61 PURE_ASCII,
62 ESC_ASCII,
63 HIGHBYTE
64 }
65
66
67 ////////////////////////////////////////////////////////////////
68 // fields
69 ////////////////////////////////////////////////////////////////
70 private InputState inputState;
71 private boolean done;
72 private boolean start;
73 private boolean gotData;
74 private byte lastChar;
75 private String detectedCharset;
76
77 private CharsetProber[] probers;
78 private CharsetProber escCharsetProber;
79
80 private CharsetListener listener;
81
82
83 ////////////////////////////////////////////////////////////////
84 // methods
85 ////////////////////////////////////////////////////////////////
86 /**
87 * @param listener a listener object that is notified of
88 * the detected encocoding. Can be null.
89 */
90 public UniversalDetector(CharsetListener listener)
91 {
92 this.listener = listener;
93 this.escCharsetProber = null;
94 this.probers = new CharsetProber[3];
95 for (int i=0; i<this.probers.length; ++i) {
96 this.probers[i] = null;
97 }
98
99 reset();
100 }
101
102 public boolean isDone()
103 {
104 return this.done;
105 }
106
107 /**
108 * @return The detected encoding is returned. If the detector couldn't
109 * determine what encoding was used, null is returned.
110 */
111 public String getDetectedCharset()
112 {
113 return this.detectedCharset;
114 }
115
116 public void setListener(CharsetListener listener)
117 {
118 this.listener = listener;
119 }
120
121 public CharsetListener getListener()
122 {
123 return this.listener;
124 }
125
126 public void handleData(final byte[] buf, int offset, int length)
127 {
128 if (this.done) {
129 return;
130 }
131
132 if (length > 0) {
133 this.gotData = true;
134 }
135
136 if (this.start) {
137 this.start = false;
138 if (length > 3) {
139 int b1 = buf[offset] & 0xFF;
140 int b2 = buf[offset+1] & 0xFF;
141 int b3 = buf[offset+2] & 0xFF;
142 int b4 = buf[offset+3] & 0xFF;
143
144 switch (b1) {
145 case 0xEF:
146 if (b2 == 0xBB && b3 == 0xBF) {
147 this.detectedCharset = Constants.CHARSET_UTF_8;
148 }
149 break;
150 case 0xFE:
151 if (b2 == 0xFF && b3 == 0x00 && b4 == 0x00) {
152 this.detectedCharset = Constants.CHARSET_X_ISO_10646_UCS_4_3412;
153 } else if (b2 == 0xFF) {
154 this.detectedCharset = Constants.CHARSET_UTF_16BE;
155 }
156 break;
157 case 0x00:
158 if (b2 == 0x00 && b3 == 0xFE && b4 == 0xFF) {
159 this.detectedCharset = Constants.CHARSET_UTF_32BE;
160 } else if (b2 == 0x00 && b3 == 0xFF && b4 == 0xFE) {
161 this.detectedCharset = Constants.CHARSET_X_ISO_10646_UCS_4_2143;
162 }
163 break;
164 case 0xFF:
165 if (b2 == 0xFE && b3 == 0x00 && b4 == 0x00) {
166 this.detectedCharset = Constants.CHARSET_UTF_32LE;
167 } else if (b2 == 0xFE) {
168 this.detectedCharset = Constants.CHARSET_UTF_16LE;
169 }
170 break;
171 } // swich end
172
173 if (this.detectedCharset != null) {
174 this.done = true;
175 return;
176 }
177 }
178 } // if (start) end
179
180 int maxPos = offset + length;
181 for (int i=offset; i<maxPos; ++i) {
182 int c = buf[i] & 0xFF;
183 if ((c & 0x80) != 0 && c != 0xA0) {
184 if (this.inputState != InputState.HIGHBYTE) {
185 this.inputState = InputState.HIGHBYTE;
186
187 if (this.escCharsetProber != null) {
188 this.escCharsetProber = null;
189 }
190
191 if (this.probers[0] == null) {
192 this.probers[0] = new MBCSGroupProber();
193 }
194 if (this.probers[1] == null) {
195 this.probers[1] = new SBCSGroupProber();
196 }
197 if (this.probers[2] == null) {
198 this.probers[2] = new Latin1Prober();
199 }
200 }
201 } else {
202 if (this.inputState == InputState.PURE_ASCII &&
203 (c == 0x1B || (c == 0x7B && this.lastChar == 0x7E))) {
204 this.inputState = InputState.ESC_ASCII;
205 }
206 this.lastChar = buf[i];
207 }
208 } // for end
209
210 CharsetProber.ProbingState st;
211 if (this.inputState == InputState.ESC_ASCII) {
212 if (this.escCharsetProber == null) {
213 this.escCharsetProber = new EscCharsetProber();
214 }
215 st = this.escCharsetProber.handleData(buf, offset, length);
216 if (st == CharsetProber.ProbingState.FOUND_IT) {
217 this.done = true;
218 this.detectedCharset = this.escCharsetProber.getCharSetName();
219 }
220 } else if (this.inputState == InputState.HIGHBYTE) {
221 for (int i=0; i<this.probers.length; ++i) {
222 st = this.probers[i].handleData(buf, offset, length);
223 if (st == CharsetProber.ProbingState.FOUND_IT) {
224 this.done = true;
225 this.detectedCharset = this.probers[i].getCharSetName();
226 return;
227 }
228 }
229 } else { // pure ascii
230 // do nothing
231 }
232 }
233
234 public void dataEnd()
235 {
236 if (!this.gotData) {
237 return;
238 }
239
240 if (this.detectedCharset != null) {
241 this.done = true;
242 if (this.listener != null) {
243 this.listener.report(this.detectedCharset);
244 }
245 return;
246 }
247
248 if (this.inputState == InputState.HIGHBYTE) {
249 float proberConfidence;
250 float maxProberConfidence = 0.0f;
251 int maxProber = 0;
252
253 for (int i=0; i<this.probers.length; ++i) {
254 proberConfidence = this.probers[i].getConfidence();
255 if (proberConfidence > maxProberConfidence) {
256 maxProberConfidence = proberConfidence;
257 maxProber = i;
258 }
259 }
260
261 if (maxProberConfidence > MINIMUM_THRESHOLD) {
262 this.detectedCharset = this.probers[maxProber].getCharSetName();
263 if (this.listener != null) {
264 this.listener.report(this.detectedCharset);
265 }
266 }
267 } else if (this.inputState == InputState.ESC_ASCII) {
268 // do nothing
269 } else {
270 // do nothing
271 }
272 }
273
274 public void reset()
275 {
276 this.done = false;
277 this.start = true;
278 this.detectedCharset = null;
279 this.gotData = false;
280 this.inputState = InputState.PURE_ASCII;
281 this.lastChar = 0;
282
283 if (this.escCharsetProber != null) {
284 this.escCharsetProber.reset();
285 }
286
287 for (int i=0; i<this.probers.length; ++i) {
288 if (this.probers[i] != null) {
289 this.probers[i].reset();
290 }
291 }
292 }
293
294
295 ////////////////////////////////////////////////////////////////
296 // testing
297 ////////////////////////////////////////////////////////////////
298 public static void main(String[] args) throws Exception
299 {
300 if (args.length != 1) {
301 System.out.println("USAGE: java UniversalDetector filename");
302 return;
303 }
304
305 UniversalDetector detector = new UniversalDetector(
306 new CharsetListener() {
307 public void report(String name)
308 {
309 System.out.println("charset = " + name);
310 }
311 }
312 );
313
314 byte[] buf = new byte[4096];
315 java.io.FileInputStream fis = new java.io.FileInputStream(args[0]);
316
317 int nread;
318 while ((nread = fis.read(buf)) > 0 && !detector.isDone()) {
319 detector.handleData(buf, 0, nread);
320 }
321 detector.dataEnd();
322 }
323 }
+0
-145
src/org/mozilla/universalchardet/prober/Big5Prober.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 * Lersh99
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import org.mozilla.universalchardet.prober.distributionanalysis.Big5DistributionAnalysis;
41 import org.mozilla.universalchardet.prober.statemachine.Big5SMModel;
42 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
43 import org.mozilla.universalchardet.prober.statemachine.SMModel;
44 import org.mozilla.universalchardet.Constants;
45
46
47 public class Big5Prober extends CharsetProber
48 {
49 ////////////////////////////////////////////////////////////////
50 // fields
51 ////////////////////////////////////////////////////////////////
52 private CodingStateMachine codingSM;
53 private ProbingState state;
54
55 private Big5DistributionAnalysis distributionAnalyzer;
56
57 private byte[] lastChar;
58
59 private static final SMModel smModel = new Big5SMModel();
60
61
62 ////////////////////////////////////////////////////////////////
63 // methods
64 ////////////////////////////////////////////////////////////////
65 public Big5Prober()
66 {
67 super();
68 this.codingSM = new CodingStateMachine(smModel);
69 this.distributionAnalyzer = new Big5DistributionAnalysis();
70 this.lastChar = new byte[2];
71 reset();
72 }
73
74 @Override
75 public String getCharSetName()
76 {
77 return Constants.CHARSET_BIG5;
78 }
79
80 @Override
81 public float getConfidence()
82 {
83 float distribCf = this.distributionAnalyzer.getConfidence();
84
85 return distribCf;
86 }
87
88 @Override
89 public ProbingState getState()
90 {
91 return this.state;
92 }
93
94 @Override
95 public ProbingState handleData(byte[] buf, int offset, int length)
96 {
97 int codingState;
98
99 int maxPos = offset + length;
100 for (int i=offset; i<maxPos; ++i) {
101 codingState = this.codingSM.nextState(buf[i]);
102 if (codingState == SMModel.ERROR) {
103 this.state = ProbingState.NOT_ME;
104 break;
105 }
106 if (codingState == SMModel.ITSME) {
107 this.state = ProbingState.FOUND_IT;
108 break;
109 }
110 if (codingState == SMModel.START) {
111 int charLen = this.codingSM.getCurrentCharLen();
112 if (i == offset) {
113 this.lastChar[1] = buf[offset];
114 this.distributionAnalyzer.handleOneChar(this.lastChar, 0, charLen);
115 } else {
116 this.distributionAnalyzer.handleOneChar(buf, i-1, charLen);
117 }
118 }
119 }
120
121 this.lastChar[0] = buf[maxPos-1];
122
123 if (this.state == ProbingState.DETECTING) {
124 if (this.distributionAnalyzer.gotEnoughData() && getConfidence() > SHORTCUT_THRESHOLD) {
125 this.state = ProbingState.FOUND_IT;
126 }
127 }
128
129 return this.state;
130 }
131
132 @Override
133 public void reset()
134 {
135 this.codingSM.reset();
136 this.state = ProbingState.DETECTING;
137 this.distributionAnalyzer.reset();
138 java.util.Arrays.fill(this.lastChar, (byte)0);
139 }
140
141 @Override
142 public void setOption()
143 {}
144 }
+0
-179
src/org/mozilla/universalchardet/prober/CharsetProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei TAKETA <k-tak@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import java.nio.ByteBuffer;
41
42
43 public abstract class CharsetProber
44 {
45 ////////////////////////////////////////////////////////////////
46 // constants
47 ////////////////////////////////////////////////////////////////
48 public static final float SHORTCUT_THRESHOLD = 0.95f;
49 public static final int ASCII_A = 0x61; // 'a'
50 public static final int ASCII_Z = 0x7A; // 'z'
51 public static final int ASCII_A_CAPITAL = 0x41; // 'A'
52 public static final int ASCII_Z_CAPITAL = 0x5A; // 'Z'
53 public static final int ASCII_LT = 0x3C; // '<'
54 public static final int ASCII_GT = 0x3E; // '>'
55 public static final int ASCII_SP = 0x20; // ' '
56
57
58 ////////////////////////////////////////////////////////////////
59 // inner types
60 ////////////////////////////////////////////////////////////////
61 public enum ProbingState
62 {
63 DETECTING,
64 FOUND_IT,
65 NOT_ME
66 }
67
68
69 ////////////////////////////////////////////////////////////////
70 // methods
71 ////////////////////////////////////////////////////////////////
72 public CharsetProber()
73 {}
74
75 public abstract String getCharSetName();
76 public abstract ProbingState handleData(final byte[] buf, int offset, int length);
77 public abstract ProbingState getState();
78 public abstract void reset();
79 public abstract float getConfidence();
80 public abstract void setOption();
81
82 // ByteBuffer.position() indicates number of bytes written.
83 public ByteBuffer filterWithoutEnglishLetters(final byte[] buf, int offset, int length)
84 {
85 ByteBuffer out = ByteBuffer.allocate(length);
86
87 boolean meetMSB = false;
88 byte c;
89
90 int prevPtr = offset;
91 int curPtr = offset;
92 int maxPtr = offset + length;
93
94 for (; curPtr<maxPtr; ++curPtr) {
95 c = buf[curPtr];
96 if (!isAscii(c)) {
97 meetMSB = true;
98 } else if (isAsciiSymbol(c)) {
99 // current char is a symbol, most likely a punctuation.
100 // we treat it as segment delimiter
101 if (meetMSB && curPtr > prevPtr) {
102 // this segment contains more than single symbol,
103 // and it has upper ASCII, we need to keep it
104 out.put(buf, prevPtr, (curPtr-prevPtr));
105 out.put((byte)ASCII_SP);
106 prevPtr = curPtr + 1;
107 meetMSB = false;
108 } else {
109 // ignore current segment.
110 // (either because it is just a symbol or just an English word)
111 prevPtr = curPtr + 1;
112 }
113 }
114 }
115
116 if (meetMSB && curPtr > prevPtr) {
117 out.put(buf, prevPtr, (curPtr-prevPtr));
118 }
119
120 return out;
121 }
122
123 public ByteBuffer filterWithEnglishLetters(final byte[] buf, int offset, int length)
124 {
125 ByteBuffer out = ByteBuffer.allocate(length);
126
127 boolean isInTag = false;
128 byte c;
129
130 int prevPtr = offset;
131 int curPtr = offset;
132 int maxPtr = offset + length;
133
134 for (; curPtr < maxPtr; ++curPtr) {
135 c = buf[curPtr];
136
137 if (c == ASCII_GT) {
138 isInTag = false;
139 } else if (c == ASCII_LT) {
140 isInTag = true;
141 }
142
143 if (isAscii(c) && isAsciiSymbol(c)) {
144 if (curPtr > prevPtr && !isInTag) {
145 // Current segment contains more than just a symbol
146 // and it is not inside a tag, keep it.
147 out.put(buf, prevPtr, (curPtr-prevPtr));
148 out.put((byte)ASCII_SP);
149 prevPtr = curPtr + 1;
150 } else {
151 prevPtr = curPtr + 1;
152 }
153 }
154 }
155
156 // If the current segment contains more than just a symbol
157 // and it is not inside a tag then keep it.
158 if (!isInTag && curPtr > prevPtr) {
159 out.put(buf, prevPtr, (curPtr-prevPtr));
160 }
161
162 return out;
163 }
164
165 private boolean isAscii(byte b)
166 {
167 return ((b & 0x80) == 0);
168 }
169
170 // b must be in ASCII code range (MSB can't be 1).
171 private boolean isAsciiSymbol(byte b)
172 {
173 int c = b & 0xFF;
174 return ((c < ASCII_A_CAPITAL) ||
175 (c > ASCII_Z_CAPITAL && c < ASCII_A) ||
176 (c > ASCII_Z));
177 }
178 }
+0
-152
src/org/mozilla/universalchardet/prober/EUCJPProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
40 import org.mozilla.universalchardet.prober.statemachine.EUCJPSMModel;
41 import org.mozilla.universalchardet.prober.statemachine.SMModel;
42 import org.mozilla.universalchardet.prober.contextanalysis.EUCJPContextAnalysis;
43 import org.mozilla.universalchardet.prober.distributionanalysis.EUCJPDistributionAnalysis;
44 import org.mozilla.universalchardet.Constants;
45
46
47 public class EUCJPProber extends CharsetProber
48 {
49 ////////////////////////////////////////////////////////////////
50 // fields
51 ////////////////////////////////////////////////////////////////
52 private CodingStateMachine codingSM;
53 private ProbingState state;
54
55 private EUCJPContextAnalysis contextAnalyzer;
56 private EUCJPDistributionAnalysis distributionAnalyzer;
57
58 private byte[] lastChar;
59
60 private static final SMModel smModel = new EUCJPSMModel();
61
62
63 ////////////////////////////////////////////////////////////////
64 // methods
65 ////////////////////////////////////////////////////////////////
66 public EUCJPProber()
67 {
68 super();
69 this.codingSM = new CodingStateMachine(smModel);
70 this.contextAnalyzer = new EUCJPContextAnalysis();
71 this.distributionAnalyzer = new EUCJPDistributionAnalysis();
72 this.lastChar = new byte[2];
73 reset();
74 }
75
76 @Override
77 public String getCharSetName()
78 {
79 return Constants.CHARSET_EUC_JP;
80 }
81
82 @Override
83 public float getConfidence()
84 {
85 float contextCf = this.contextAnalyzer.getConfidence();
86 float distribCf = this.distributionAnalyzer.getConfidence();
87
88 return Math.max(contextCf, distribCf);
89 }
90
91 @Override
92 public ProbingState getState()
93 {
94 return this.state;
95 }
96
97 @Override
98 public ProbingState handleData(byte[] buf, int offset, int length)
99 {
100 int codingState;
101
102 int maxPos = offset + length;
103 for (int i=offset; i<maxPos; ++i) {
104 codingState = this.codingSM.nextState(buf[i]);
105 if (codingState == SMModel.ERROR) {
106 this.state = ProbingState.NOT_ME;
107 break;
108 }
109 if (codingState == SMModel.ITSME) {
110 this.state = ProbingState.FOUND_IT;
111 break;
112 }
113 if (codingState == SMModel.START) {
114 int charLen = this.codingSM.getCurrentCharLen();
115
116 if (i == offset) {
117 this.lastChar[1] = buf[offset];
118 this.contextAnalyzer.handleOneChar(this.lastChar, 0, charLen);
119 this.distributionAnalyzer.handleOneChar(this.lastChar, 0, charLen);
120 } else {
121 this.contextAnalyzer.handleOneChar(buf, i-1, charLen);
122 this.distributionAnalyzer.handleOneChar(buf, i-1, charLen);
123 }
124 }
125 }
126
127 this.lastChar[0] = buf[maxPos-1];
128
129 if (this.state == ProbingState.DETECTING) {
130 if (this.contextAnalyzer.gotEnoughData() && getConfidence() > SHORTCUT_THRESHOLD) {
131 this.state = ProbingState.FOUND_IT;
132 }
133 }
134
135 return this.state;
136 }
137
138 @Override
139 public void reset()
140 {
141 this.codingSM.reset();
142 this.state = ProbingState.DETECTING;
143 this.contextAnalyzer.reset();
144 this.distributionAnalyzer.reset();
145 java.util.Arrays.fill(this.lastChar, (byte)0);
146 }
147
148 @Override
149 public void setOption()
150 {}
151 }
+0
-144
src/org/mozilla/universalchardet/prober/EUCKRProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.prober.distributionanalysis.EUCKRDistributionAnalysis;
40 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
41 import org.mozilla.universalchardet.prober.statemachine.EUCKRSMModel;
42 import org.mozilla.universalchardet.prober.statemachine.SMModel;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class EUCKRProber extends CharsetProber
47 {
48 ////////////////////////////////////////////////////////////////
49 // fields
50 ////////////////////////////////////////////////////////////////
51 private CodingStateMachine codingSM;
52 private ProbingState state;
53
54 private EUCKRDistributionAnalysis distributionAnalyzer;
55
56 private byte[] lastChar;
57
58 private static final SMModel smModel = new EUCKRSMModel();
59
60
61 ////////////////////////////////////////////////////////////////
62 // methods
63 ////////////////////////////////////////////////////////////////
64 public EUCKRProber()
65 {
66 super();
67 this.codingSM = new CodingStateMachine(smModel);
68 this.distributionAnalyzer = new EUCKRDistributionAnalysis();
69 this.lastChar = new byte[2];
70 reset();
71 }
72
73 @Override
74 public String getCharSetName()
75 {
76 return Constants.CHARSET_EUC_KR;
77 }
78
79 @Override
80 public float getConfidence()
81 {
82 float distribCf = this.distributionAnalyzer.getConfidence();
83
84 return distribCf;
85 }
86
87 @Override
88 public ProbingState getState()
89 {
90 return this.state;
91 }
92
93 @Override
94 public ProbingState handleData(byte[] buf, int offset, int length)
95 {
96 int codingState;
97
98 int maxPos = offset + length;
99 for (int i=offset; i<maxPos; ++i) {
100 codingState = codingSM.nextState(buf[i]);
101 if (codingState == SMModel.ERROR) {
102 this.state = ProbingState.NOT_ME;
103 break;
104 }
105 if (codingState == SMModel.ITSME) {
106 this.state = ProbingState.FOUND_IT;
107 break;
108 }
109 if (codingState == SMModel.START) {
110 int charLen = this.codingSM.getCurrentCharLen();
111 if (i == offset) {
112 this.lastChar[1] = buf[offset];
113 this.distributionAnalyzer.handleOneChar(this.lastChar, 0, charLen);
114 } else {
115 this.distributionAnalyzer.handleOneChar(buf, i-1, charLen);
116 }
117 }
118 }
119
120 this.lastChar[0] = buf[maxPos-1];
121
122 if (this.state == ProbingState.DETECTING) {
123 if (this.distributionAnalyzer.gotEnoughData() && getConfidence() > SHORTCUT_THRESHOLD) {
124 this.state = ProbingState.FOUND_IT;
125 }
126 }
127
128 return this.state;
129 }
130
131 @Override
132 public void reset()
133 {
134 this.codingSM.reset();
135 this.state = ProbingState.DETECTING;
136 this.distributionAnalyzer.reset();
137 java.util.Arrays.fill(this.lastChar, (byte)0);
138 }
139
140 @Override
141 public void setOption()
142 {}
143 }
+0
-145
src/org/mozilla/universalchardet/prober/EUCTWProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 * Lersh99
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import org.mozilla.universalchardet.prober.distributionanalysis.EUCTWDistributionAnalysis;
41 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
42 import org.mozilla.universalchardet.prober.statemachine.EUCTWSMModel;
43 import org.mozilla.universalchardet.prober.statemachine.SMModel;
44 import org.mozilla.universalchardet.Constants;
45
46
47 public class EUCTWProber extends CharsetProber
48 {
49 ////////////////////////////////////////////////////////////////
50 // fields
51 ////////////////////////////////////////////////////////////////
52 private CodingStateMachine codingSM;
53 private ProbingState state;
54
55 private EUCTWDistributionAnalysis distributionAnalyzer;
56
57 private byte[] lastChar;
58
59 private static final SMModel smModel = new EUCTWSMModel();
60
61
62 ////////////////////////////////////////////////////////////////
63 // methods
64 ////////////////////////////////////////////////////////////////
65 public EUCTWProber()
66 {
67 super();
68 this.codingSM = new CodingStateMachine(smModel);
69 this.distributionAnalyzer = new EUCTWDistributionAnalysis();
70 this.lastChar = new byte[2];
71 reset();
72 }
73
74 @Override
75 public String getCharSetName()
76 {
77 return Constants.CHARSET_EUC_TW;
78 }
79
80 @Override
81 public float getConfidence()
82 {
83 float distribCf = this.distributionAnalyzer.getConfidence();
84
85 return distribCf;
86 }
87
88 @Override
89 public ProbingState getState()
90 {
91 return this.state;
92 }
93
94 @Override
95 public ProbingState handleData(byte[] buf, int offset, int length)
96 {
97 int codingState;
98
99 int maxPos = offset + length;
100 for (int i=offset; i<maxPos; ++i) {
101 codingState = this.codingSM.nextState(buf[i]);
102 if (codingState == SMModel.ERROR) {
103 this.state = ProbingState.NOT_ME;
104 break;
105 }
106 if (codingState == SMModel.ITSME) {
107 this.state = ProbingState.FOUND_IT;
108 break;
109 }
110 if (codingState == SMModel.START) {
111 int charLen = this.codingSM.getCurrentCharLen();
112 if (i == offset) {
113 this.lastChar[1] = buf[offset];
114 this.distributionAnalyzer.handleOneChar(this.lastChar, 0, charLen);
115 } else {
116 this.distributionAnalyzer.handleOneChar(buf, i-1, charLen);
117 }
118 }
119 }
120
121 this.lastChar[0] = buf[maxPos-1];
122
123 if (this.state == ProbingState.DETECTING) {
124 if (this.distributionAnalyzer.gotEnoughData() && getConfidence() > SHORTCUT_THRESHOLD) {
125 this.state = ProbingState.FOUND_IT;
126 }
127 }
128
129 return this.state;
130 }
131
132 @Override
133 public void reset()
134 {
135 this.codingSM.reset();
136 this.state = ProbingState.DETECTING;
137 this.distributionAnalyzer.reset();
138 java.util.Arrays.fill(this.lastChar, (byte)0);
139 }
140
141 @Override
142 public void setOption()
143 {}
144 }
+0
-143
src/org/mozilla/universalchardet/prober/EscCharsetProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
40 import org.mozilla.universalchardet.prober.statemachine.SMModel;
41 import org.mozilla.universalchardet.prober.statemachine.HZSMModel;
42 import org.mozilla.universalchardet.prober.statemachine.ISO2022CNSMModel;
43 import org.mozilla.universalchardet.prober.statemachine.ISO2022JPSMModel;
44 import org.mozilla.universalchardet.prober.statemachine.ISO2022KRSMModel;
45
46
47 public class EscCharsetProber extends CharsetProber
48 {
49 ////////////////////////////////////////////////////////////////
50 // fields
51 ////////////////////////////////////////////////////////////////
52 private CodingStateMachine[] codingSM;
53 private int activeSM;
54 private ProbingState state;
55 private String detectedCharset;
56
57 private static final HZSMModel hzsModel = new HZSMModel();
58 private static final ISO2022CNSMModel iso2022cnModel = new ISO2022CNSMModel();
59 private static final ISO2022JPSMModel iso2022jpModel = new ISO2022JPSMModel();
60 private static final ISO2022KRSMModel iso2022krModel = new ISO2022KRSMModel();
61
62
63 ////////////////////////////////////////////////////////////////
64 // methods
65 ////////////////////////////////////////////////////////////////
66 public EscCharsetProber()
67 {
68 super();
69
70 this.codingSM = new CodingStateMachine[4];
71 this.codingSM[0] = new CodingStateMachine(hzsModel);
72 this.codingSM[1] = new CodingStateMachine(iso2022cnModel);
73 this.codingSM[2] = new CodingStateMachine(iso2022jpModel);
74 this.codingSM[3] = new CodingStateMachine(iso2022krModel);
75
76 reset();
77 }
78
79 @Override
80 public String getCharSetName()
81 {
82 return this.detectedCharset;
83 }
84
85 @Override
86 public float getConfidence()
87 {
88 return 0.99f;
89 }
90
91 @Override
92 public ProbingState getState()
93 {
94 return this.state;
95 }
96
97 @Override
98 public ProbingState handleData(byte[] buf, int offset, int length)
99 {
100 int codingState;
101
102 int maxPos = offset + length;
103 for (int i=offset; i<maxPos && this.state==ProbingState.DETECTING; ++i) {
104 for (int j=this.activeSM-1; j>=0; --j) {
105 codingState = this.codingSM[j].nextState(buf[i]);
106 if (codingState == SMModel.ERROR) {
107 --this.activeSM;
108 if (this.activeSM <= 0) {
109 this.state = ProbingState.NOT_ME;
110 return this.state;
111 } else if (j != this.activeSM) {
112 CodingStateMachine t;
113 t = this.codingSM[this.activeSM];
114 this.codingSM[this.activeSM] = this.codingSM[j];
115 this.codingSM[j] = t;
116 }
117 } else if (codingState == SMModel.ITSME) {
118 this.state = ProbingState.FOUND_IT;
119 this.detectedCharset = this.codingSM[j].getCodingStateMachine();
120 return this.state;
121 }
122 }
123 }
124
125 return this.state;
126 }
127
128 @Override
129 public void reset()
130 {
131 this.state = ProbingState.DETECTING;
132 for (int i=0; i<this.codingSM.length; ++i) {
133 this.codingSM[i].reset();
134 }
135 this.activeSM = this.codingSM.length;
136 this.detectedCharset = null;
137 }
138
139 @Override
140 public void setOption()
141 {}
142 }
+0
-145
src/org/mozilla/universalchardet/prober/GB18030Prober.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 * Lersh99
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import org.mozilla.universalchardet.prober.distributionanalysis.GB2312DistributionAnalysis;
41 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
42 import org.mozilla.universalchardet.prober.statemachine.GB18030SMModel;
43 import org.mozilla.universalchardet.prober.statemachine.SMModel;
44 import org.mozilla.universalchardet.Constants;
45
46
47 public class GB18030Prober extends CharsetProber
48 {
49 ////////////////////////////////////////////////////////////////
50 // fields
51 ////////////////////////////////////////////////////////////////
52 private CodingStateMachine codingSM;
53 private ProbingState state;
54
55 private GB2312DistributionAnalysis distributionAnalyzer;
56
57 private byte[] lastChar;
58
59 private static final SMModel smModel = new GB18030SMModel();
60
61
62 ////////////////////////////////////////////////////////////////
63 // methods
64 ////////////////////////////////////////////////////////////////
65 public GB18030Prober()
66 {
67 super();
68 this.codingSM = new CodingStateMachine(smModel);
69 this.distributionAnalyzer = new GB2312DistributionAnalysis();
70 this.lastChar = new byte[2];
71 reset();
72 }
73
74 @Override
75 public String getCharSetName()
76 {
77 return Constants.CHARSET_GB18030;
78 }
79
80 @Override
81 public float getConfidence()
82 {
83 float distribCf = this.distributionAnalyzer.getConfidence();
84
85 return distribCf;
86 }
87
88 @Override
89 public ProbingState getState()
90 {
91 return this.state;
92 }
93
94 @Override
95 public ProbingState handleData(byte[] buf, int offset, int length)
96 {
97 int codingState;
98
99 int maxPos = offset + length;
100 for (int i=offset; i<maxPos; ++i) {
101 codingState = this.codingSM.nextState(buf[i]);
102 if (codingState == SMModel.ERROR) {
103 this.state = ProbingState.NOT_ME;
104 break;
105 }
106 if (codingState == SMModel.ITSME) {
107 this.state = ProbingState.FOUND_IT;
108 break;
109 }
110 if (codingState == SMModel.START) {
111 int charLen = this.codingSM.getCurrentCharLen();
112 if (i == offset) {
113 this.lastChar[1] = buf[offset];
114 this.distributionAnalyzer.handleOneChar(this.lastChar, 0, charLen);
115 } else {
116 this.distributionAnalyzer.handleOneChar(buf, i-1, charLen);
117 }
118 }
119 }
120
121 this.lastChar[0] = buf[maxPos-1];
122
123 if (this.state == ProbingState.DETECTING) {
124 if (this.distributionAnalyzer.gotEnoughData() && getConfidence() > SHORTCUT_THRESHOLD) {
125 this.state = ProbingState.FOUND_IT;
126 }
127 }
128
129 return this.state;
130 }
131
132 @Override
133 public void reset()
134 {
135 this.codingSM.reset();
136 this.state = ProbingState.DETECTING;
137 this.distributionAnalyzer.reset();
138 java.util.Arrays.fill(this.lastChar, (byte)0);
139 }
140
141 @Override
142 public void setOption()
143 {}
144 }
+0
-223
src/org/mozilla/universalchardet/prober/HebrewProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Shy Shalom <shooshX@gmail.com>
17 * Portions created by the Initial Developer are Copyright (C) 2005
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class HebrewProber extends CharsetProber
43 {
44 ////////////////////////////////////////////////////////////////
45 // fields
46 ////////////////////////////////////////////////////////////////
47 public static final int FINAL_KAF = 0xEA;
48 public static final int NORMAL_KAF = 0xEB;
49 public static final int FINAL_MEM = 0xED;
50 public static final int NORMAL_MEM = 0xEE;
51 public static final int FINAL_NUN = 0xEF;
52 public static final int NORMAL_NUN = 0xF0;
53 public static final int FINAL_PE = 0xF3;
54 public static final int NORMAL_PE = 0xF4;
55 public static final int FINAL_TSADI = 0xF5;
56 public static final int NORMAL_TSADI= 0xF6;
57
58 public static final byte SPACE = 0x20;
59
60 public static final int MIN_FINAL_CHAR_DISTANCE = 5;
61 public static final float MIN_MODEL_DISTANCE = 0.01f;
62
63
64 ////////////////////////////////////////////////////////////////
65 // fields
66 ////////////////////////////////////////////////////////////////
67 private int finalCharLogicalScore;
68 private int finalCharVisualScore;
69 private byte prev;
70 private byte beforePrev;
71
72 private CharsetProber logicalProber;
73 private CharsetProber visualProber;
74
75
76 ////////////////////////////////////////////////////////////////
77 // methods
78 ////////////////////////////////////////////////////////////////
79 public HebrewProber()
80 {
81 super();
82 this.logicalProber = null;
83 this.visualProber = null;
84 reset();
85 }
86
87 public void setModalProbers(CharsetProber logicalProber, CharsetProber visualProber)
88 {
89 this.logicalProber = logicalProber;
90 this.visualProber = visualProber;
91 }
92
93 @Override
94 public String getCharSetName()
95 {
96 // If the final letter score distance is dominant enough, rely on it.
97 int finalsub = this.finalCharLogicalScore - this.finalCharVisualScore;
98 if (finalsub >= MIN_FINAL_CHAR_DISTANCE) {
99 return Constants.CHARSET_WINDOWS_1255;
100 }
101 if (finalsub <= -MIN_FINAL_CHAR_DISTANCE) {
102 return Constants.CHARSET_ISO_8859_8;
103 }
104
105 // It's not dominant enough, try to rely on the model scores instead.
106 float modelsub = this.logicalProber.getConfidence() - this.visualProber.getConfidence();
107 if (modelsub > MIN_MODEL_DISTANCE) {
108 return Constants.CHARSET_WINDOWS_1255;
109 }
110 if (modelsub < -MIN_MODEL_DISTANCE) {
111 return Constants.CHARSET_ISO_8859_8;
112 }
113
114 // Still no good, back to final letter distance, maybe it'll save the day.
115 if (finalsub < 0) {
116 return Constants.CHARSET_ISO_8859_8;
117 }
118
119 // (finalsub > 0 - Logical) or (don't know what to do) default to Logical.
120 return Constants.CHARSET_WINDOWS_1255;
121 }
122
123 @Override
124 public float getConfidence()
125 {
126 return 0.0f;
127 }
128
129 @Override
130 public ProbingState getState()
131 {
132 // Remain active as long as any of the model probers are active.
133 if ((this.logicalProber.getState() == ProbingState.NOT_ME) &&
134 (this.visualProber.getState() == ProbingState.NOT_ME)) {
135 return ProbingState.NOT_ME;
136 }
137
138 return ProbingState.DETECTING;
139 }
140
141 @Override
142 public ProbingState handleData(byte[] buf, int offset, int length)
143 {
144 if (getState() == ProbingState.NOT_ME) {
145 return ProbingState.NOT_ME;
146 }
147
148 byte c;
149 int maxPos = offset + length;
150 for (int i=offset; i<maxPos; ++i) {
151 c = buf[i];
152 if (c == SPACE) {
153 if (this.beforePrev != SPACE) {
154 if (isFinal(this.prev)) {
155 ++this.finalCharLogicalScore;
156 } else if (isNonFinal(this.prev)) {
157 ++this.finalCharVisualScore;
158 }
159 }
160 } else {
161 if ((this.beforePrev == SPACE) &&
162 isFinal(this.prev) &&
163 c != SPACE) {
164 ++this.finalCharVisualScore;
165 }
166 }
167 this.beforePrev = this.prev;
168 this.prev = c;
169 }
170
171 return ProbingState.DETECTING;
172 }
173
174 @Override
175 public void reset()
176 {
177 this.finalCharLogicalScore = 0;
178 this.finalCharVisualScore = 0;
179
180 // mPrev and mBeforePrev are initialized to space in order to simulate a word
181 // delimiter at the beginning of the data
182 this.prev = SPACE;
183 this.beforePrev = SPACE;
184 }
185
186 @Override
187 public void setOption()
188 {}
189
190 protected static boolean isFinal(byte b)
191 {
192 int c = b & 0xFF;
193 return (
194 c == FINAL_KAF ||
195 c == FINAL_MEM ||
196 c == FINAL_NUN ||
197 c == FINAL_PE ||
198 c == FINAL_TSADI
199 );
200 }
201
202 protected static boolean isNonFinal(byte b)
203 {
204 int c = b & 0xFF;
205 return (
206 c == NORMAL_KAF ||
207 c == NORMAL_MEM ||
208 c == NORMAL_NUN ||
209 c == NORMAL_PE
210 );
211 // The normal Tsadi is not a good Non-Final letter due to words like
212 // 'lechotet' (to chat) containing an apostrophe after the tsadi. This
213 // apostrophe is converted to a space in FilterWithoutEnglishLetters causing
214 // the Non-Final tsadi to appear at an end of a word even though this is not
215 // the case in the original text.
216 // The letters Pe and Kaf rarely display a related behavior of not being a
217 // good Non-Final letter. Words like 'Pop', 'Winamp' and 'Mubarak' for
218 // example legally end with a Non-Final Pe or Kaf. However, the benefit of
219 // these letters as Non-Final letters outweighs the damage since these words
220 // are quite rare.
221 }
222 }
+0
-215
src/org/mozilla/universalchardet/prober/Latin1Prober.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei Taketa <k-tak@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import java.nio.ByteBuffer;
41 import org.mozilla.universalchardet.Constants;
42
43
44 public class Latin1Prober extends CharsetProber
45 {
46 ////////////////////////////////////////////////////////////////
47 // constants
48 ////////////////////////////////////////////////////////////////
49 public static final byte UDF = 0;
50 public static final byte OTH = 1;
51 public static final byte ASC = 2;
52 public static final byte ASS = 3;
53 public static final byte ACV = 4;
54 public static final byte ACO = 5;
55 public static final byte ASV = 6;
56 public static final byte ASO = 7;
57 public static final int CLASS_NUM = 8;
58 public static final int FREQ_CAT_NUM = 4;
59
60
61 ////////////////////////////////////////////////////////////////
62 // fields
63 ////////////////////////////////////////////////////////////////
64 private ProbingState state;
65 private byte lastCharClass;
66 private int[] freqCounter;
67
68
69 ////////////////////////////////////////////////////////////////
70 // methods
71 ////////////////////////////////////////////////////////////////
72 public Latin1Prober()
73 {
74 super();
75
76 this.freqCounter = new int[FREQ_CAT_NUM];
77
78 reset();
79 }
80
81 @Override
82 public String getCharSetName()
83 {
84 return Constants.CHARSET_WINDOWS_1252;
85 }
86
87 @Override
88 public float getConfidence()
89 {
90 if (this.state == ProbingState.NOT_ME) {
91 return 0.01f;
92 }
93
94 float confidence;
95 int total = 0;
96 for (int i=0; i<this.freqCounter.length; ++i) {
97 total += this.freqCounter[i];
98 }
99
100 if (total <= 0) {
101 confidence = 0.0f;
102 } else {
103 confidence = this.freqCounter[3] * 1.0f / total;
104 confidence -= this.freqCounter[1] * 20.0f / total;
105 }
106
107 if (confidence < 0.0f) {
108 confidence = 0.0f;
109 }
110
111 // lower the confidence of latin1 so that other more accurate detector
112 // can take priority.
113 confidence *= 0.50f;
114
115 return confidence;
116 }
117
118 @Override
119 public ProbingState getState()
120 {
121 return this.state;
122 }
123
124 @Override
125 public ProbingState handleData(byte[] buf, int offset, int length)
126 {
127 ByteBuffer newBufTmp = filterWithEnglishLetters(buf, offset, length);
128
129 byte charClass;
130 byte freq;
131
132 byte[] newBuf = newBufTmp.array();
133 int newBufLen = newBufTmp.position();
134
135 for (int i=0; i<newBufLen; ++i) {
136 int c = newBuf[i] & 0xFF;
137 charClass = latin1CharToClass[c];
138 freq = latin1ClassModel[this.lastCharClass * CLASS_NUM + charClass];
139 if (freq == 0) {
140 this.state = ProbingState.NOT_ME;
141 break;
142 }
143 ++this.freqCounter[freq];
144 this.lastCharClass = charClass;
145 }
146
147 return this.state;
148 }
149
150 @Override
151 public void reset()
152 {
153 this.state = ProbingState.DETECTING;
154 this.lastCharClass = OTH;
155 for (int i=0; i<this.freqCounter.length; ++i) {
156 this.freqCounter[i] = 0;
157 }
158 }
159
160 @Override
161 public void setOption()
162 {}
163
164
165 ////////////////////////////////////////////////////////////////
166 // constants continued
167 ////////////////////////////////////////////////////////////////
168 private static final byte[] latin1CharToClass = new byte[] {
169 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 00 - 07
170 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 08 - 0F
171 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 10 - 17
172 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 18 - 1F
173 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 20 - 27
174 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 28 - 2F
175 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 30 - 37
176 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 38 - 3F
177 OTH, ASC, ASC, ASC, ASC, ASC, ASC, ASC, // 40 - 47
178 ASC, ASC, ASC, ASC, ASC, ASC, ASC, ASC, // 48 - 4F
179 ASC, ASC, ASC, ASC, ASC, ASC, ASC, ASC, // 50 - 57
180 ASC, ASC, ASC, OTH, OTH, OTH, OTH, OTH, // 58 - 5F
181 OTH, ASS, ASS, ASS, ASS, ASS, ASS, ASS, // 60 - 67
182 ASS, ASS, ASS, ASS, ASS, ASS, ASS, ASS, // 68 - 6F
183 ASS, ASS, ASS, ASS, ASS, ASS, ASS, ASS, // 70 - 77
184 ASS, ASS, ASS, OTH, OTH, OTH, OTH, OTH, // 78 - 7F
185 OTH, UDF, OTH, ASO, OTH, OTH, OTH, OTH, // 80 - 87
186 OTH, OTH, ACO, OTH, ACO, UDF, ACO, UDF, // 88 - 8F
187 UDF, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // 90 - 97
188 OTH, OTH, ASO, OTH, ASO, UDF, ASO, ACO, // 98 - 9F
189 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // A0 - A7
190 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // A8 - AF
191 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // B0 - B7
192 OTH, OTH, OTH, OTH, OTH, OTH, OTH, OTH, // B8 - BF
193 ACV, ACV, ACV, ACV, ACV, ACV, ACO, ACO, // C0 - C7
194 ACV, ACV, ACV, ACV, ACV, ACV, ACV, ACV, // C8 - CF
195 ACO, ACO, ACV, ACV, ACV, ACV, ACV, OTH, // D0 - D7
196 ACV, ACV, ACV, ACV, ACV, ACO, ACO, ACO, // D8 - DF
197 ASV, ASV, ASV, ASV, ASV, ASV, ASO, ASO, // E0 - E7
198 ASV, ASV, ASV, ASV, ASV, ASV, ASV, ASV, // E8 - EF
199 ASO, ASO, ASV, ASV, ASV, ASV, ASV, OTH, // F0 - F7
200 ASV, ASV, ASV, ASV, ASV, ASO, ASO, ASO, // F8 - FF
201 };
202
203 private static final byte[] latin1ClassModel = new byte[] {
204 /* UDF OTH ASC ASS ACV ACO ASV ASO */
205 /*UDF*/ 0, 0, 0, 0, 0, 0, 0, 0,
206 /*OTH*/ 0, 3, 3, 3, 3, 3, 3, 3,
207 /*ASC*/ 0, 3, 3, 3, 3, 3, 3, 3,
208 /*ASS*/ 0, 3, 3, 3, 1, 1, 3, 3,
209 /*ACV*/ 0, 3, 3, 3, 1, 2, 1, 2,
210 /*ACO*/ 0, 3, 3, 3, 3, 3, 3, 3,
211 /*ASV*/ 0, 3, 1, 3, 1, 1, 1, 3,
212 /*ASO*/ 0, 3, 1, 3, 1, 1, 3, 3,
213 };
214 }
+0
-181
src/org/mozilla/universalchardet/prober/MBCSGroupProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei TAKETA <k-tak@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40
41 public class MBCSGroupProber extends CharsetProber
42 {
43 ////////////////////////////////////////////////////////////////
44 // fields
45 ////////////////////////////////////////////////////////////////
46 private ProbingState state;
47 private CharsetProber[] probers;
48 private boolean[] isActive;
49 private int bestGuess;
50 private int activeNum;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public MBCSGroupProber()
57 {
58 super();
59
60 this.probers = new CharsetProber[7];
61 this.isActive = new boolean[7];
62
63 this.probers[0] = new UTF8Prober();
64 this.probers[1] = new SJISProber();
65 this.probers[2] = new EUCJPProber();
66 this.probers[3] = new GB18030Prober();
67 this.probers[4] = new EUCKRProber();
68 this.probers[5] = new Big5Prober();
69 this.probers[6] = new EUCTWProber();
70
71 reset();
72 }
73
74 @Override
75 public String getCharSetName()
76 {
77 if (this.bestGuess == -1) {
78 getConfidence();
79 if (this.bestGuess == -1) {
80 this.bestGuess = 0;
81 }
82 }
83 return this.probers[this.bestGuess].getCharSetName();
84 }
85
86 @Override
87 public float getConfidence()
88 {
89 float bestConf = 0.0f;
90 float cf;
91
92 if (this.state == ProbingState.FOUND_IT) {
93 return 0.99f;
94 } else if (this.state == ProbingState.NOT_ME) {
95 return 0.01f;
96 } else {
97 for (int i=0; i<probers.length; ++i) {
98 if (!this.isActive[i]) {
99 continue;
100 }
101
102 cf = this.probers[i].getConfidence();
103 if (bestConf < cf) {
104 bestConf = cf;
105 this.bestGuess = i;
106 }
107 }
108 }
109
110 return bestConf;
111 }
112
113 @Override
114 public ProbingState getState()
115 {
116 return this.state;
117 }
118
119 @Override
120 public ProbingState handleData(byte[] buf, int offset, int length)
121 {
122 ProbingState st;
123
124 boolean keepNext = true;
125 byte[] highbyteBuf = new byte[length];
126 int highpos = 0;
127
128 int maxPos = offset + length;
129 for (int i=offset; i<maxPos; ++i) {
130 if ((buf[i] & 0x80) != 0) {
131 highbyteBuf[highpos++] = buf[i];
132 keepNext = true;
133 } else {
134 //if previous is highbyte, keep this even it is a ASCII
135 if (keepNext) {
136 highbyteBuf[highpos++] = buf[i];
137 keepNext = false;
138 }
139 }
140 }
141
142 for (int i=0; i<this.probers.length; ++i) {
143 if (!this.isActive[i]) {
144 continue;
145 }
146 st = this.probers[i].handleData(highbyteBuf, 0, highpos);
147 if (st == ProbingState.FOUND_IT) {
148 this.bestGuess = i;
149 this.state = ProbingState.FOUND_IT;
150 break;
151 } else if (st == ProbingState.NOT_ME) {
152 this.isActive[i] = false;
153 --this.activeNum;
154 if (this.activeNum <= 0) {
155 this.state = ProbingState.NOT_ME;
156 break;
157 }
158 }
159 }
160
161 return this.state;
162 }
163
164 @Override
165 public void reset()
166 {
167 this.activeNum = 0;
168 for (int i=0; i<this.probers.length; ++i) {
169 this.probers[i].reset();
170 this.isActive[i] = true;
171 ++this.activeNum;
172 }
173 this.bestGuess = -1;
174 this.state = ProbingState.DETECTING;
175 }
176
177 @Override
178 public void setOption()
179 {}
180 }
+0
-208
src/org/mozilla/universalchardet/prober/SBCSGroupProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei TAKETA <k-tak@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.prober.sequence.SequenceModel;
40 import org.mozilla.universalchardet.prober.sequence.Win1251Model;
41 import org.mozilla.universalchardet.prober.sequence.Koi8rModel;
42 import org.mozilla.universalchardet.prober.sequence.Latin5Model;
43 import org.mozilla.universalchardet.prober.sequence.MacCyrillicModel;
44 import org.mozilla.universalchardet.prober.sequence.Ibm866Model;
45 import org.mozilla.universalchardet.prober.sequence.Ibm855Model;
46 import org.mozilla.universalchardet.prober.sequence.Latin7Model;
47 import org.mozilla.universalchardet.prober.sequence.Win1253Model;
48 import org.mozilla.universalchardet.prober.sequence.Latin5BulgarianModel;
49 import org.mozilla.universalchardet.prober.sequence.Win1251BulgarianModel;
50 import org.mozilla.universalchardet.prober.sequence.HebrewModel;
51
52 import java.nio.ByteBuffer;
53
54 public class SBCSGroupProber extends CharsetProber
55 {
56 ////////////////////////////////////////////////////////////////
57 // fields
58 ////////////////////////////////////////////////////////////////
59 private ProbingState state;
60 private CharsetProber[] probers;
61 private boolean[] isActive;
62 private int bestGuess;
63 private int activeNum;
64
65
66 // models
67 private static final SequenceModel win1251Model = new Win1251Model();
68 private static final SequenceModel koi8rModel = new Koi8rModel();
69 private static final SequenceModel latin5Model = new Latin5Model();
70 private static final SequenceModel macCyrillicModel = new MacCyrillicModel();
71 private static final SequenceModel ibm866Model = new Ibm866Model();
72 private static final SequenceModel ibm855Model = new Ibm855Model();
73 private static final SequenceModel latin7Model = new Latin7Model();
74 private static final SequenceModel win1253Model = new Win1253Model();
75 private static final SequenceModel latin5BulgarianModel = new Latin5BulgarianModel();
76 private static final SequenceModel win1251BulgarianModel = new Win1251BulgarianModel();
77 private static final SequenceModel hebrewModel = new HebrewModel();
78
79
80 ////////////////////////////////////////////////////////////////
81 // methods
82 ////////////////////////////////////////////////////////////////
83 public SBCSGroupProber()
84 {
85 super();
86
87 this.probers = new CharsetProber[13];
88 this.isActive = new boolean[13];
89
90 this.probers[0] = new SingleByteCharsetProber(win1251Model);
91 this.probers[1] = new SingleByteCharsetProber(koi8rModel);
92 this.probers[2] = new SingleByteCharsetProber(latin5Model);
93 this.probers[3] = new SingleByteCharsetProber(macCyrillicModel);
94 this.probers[4] = new SingleByteCharsetProber(ibm866Model);
95 this.probers[5] = new SingleByteCharsetProber(ibm855Model);
96 this.probers[6] = new SingleByteCharsetProber(latin7Model);
97 this.probers[7] = new SingleByteCharsetProber(win1253Model);
98 this.probers[8] = new SingleByteCharsetProber(latin5BulgarianModel);
99 this.probers[9] = new SingleByteCharsetProber(win1251BulgarianModel);
100
101 HebrewProber hebprober = new HebrewProber();
102 this.probers[10] = hebprober;
103 this.probers[11] = new SingleByteCharsetProber(hebrewModel, false, hebprober);
104 this.probers[12] = new SingleByteCharsetProber(hebrewModel, true, hebprober);
105 hebprober.setModalProbers(this.probers[11], this.probers[12]);
106
107 reset();
108 }
109
110 @Override
111 public String getCharSetName()
112 {
113 if (this.bestGuess == -1) {
114 getConfidence();
115 if (this.bestGuess == -1) {
116 this.bestGuess = 0;
117 }
118 }
119
120 return this.probers[this.bestGuess].getCharSetName();
121 }
122
123 @Override
124 public float getConfidence()
125 {
126 float bestConf = 0.0f;
127 float cf;
128
129 if (this.state == ProbingState.FOUND_IT) {
130 return 0.99f;
131 } else if (this.state == ProbingState.NOT_ME) {
132 return 0.01f;
133 } else {
134 for (int i=0; i<probers.length; ++i) {
135 if (!this.isActive[i]) {
136 continue;
137 }
138
139 cf = this.probers[i].getConfidence();
140 if (bestConf < cf) {
141 bestConf = cf;
142 this.bestGuess = i;
143 }
144 }
145 }
146
147 return bestConf;
148 }
149
150 @Override
151 public ProbingState getState()
152 {
153 return this.state;
154 }
155
156 @Override
157 public ProbingState handleData(byte[] buf, int offset, int length)
158 {
159 ProbingState st;
160
161 do {
162 ByteBuffer newbuf = filterWithoutEnglishLetters(buf, offset, length);
163 if (newbuf.position() == 0) {
164 break;
165 }
166
167 for (int i=0; i<this.probers.length; ++i) {
168 if (!this.isActive[i]) {
169 continue;
170 }
171 st = this.probers[i].handleData(newbuf.array(), 0, newbuf.position());
172 if (st == ProbingState.FOUND_IT) {
173 this.bestGuess = i;
174 this.state = ProbingState.FOUND_IT;
175 break;
176 } else if (st == ProbingState.NOT_ME) {
177 this.isActive[i] = false;
178 --this.activeNum;
179 if (this.activeNum <= 0) {
180 this.state = ProbingState.NOT_ME;
181 break;
182 }
183 }
184 }
185 } while (false);
186
187 return this.state;
188 }
189
190 @Override
191 public void reset()
192 {
193 this.activeNum = 0;
194 for (int i=0; i<this.probers.length; ++i) {
195 this.probers[i].reset();
196 this.isActive[i] = true;
197 ++this.activeNum;
198 }
199
200 this.bestGuess = -1;
201 this.state = ProbingState.DETECTING;
202 }
203
204 @Override
205 public void setOption()
206 {}
207 }
+0
-151
src/org/mozilla/universalchardet/prober/SJISProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
40 import org.mozilla.universalchardet.prober.statemachine.SJISSMModel;
41 import org.mozilla.universalchardet.prober.statemachine.SMModel;
42 import org.mozilla.universalchardet.prober.contextanalysis.SJISContextAnalysis;
43 import org.mozilla.universalchardet.prober.distributionanalysis.SJISDistributionAnalysis;
44 import org.mozilla.universalchardet.Constants;
45
46
47 public class SJISProber extends CharsetProber
48 {
49 ////////////////////////////////////////////////////////////////
50 // fields
51 ////////////////////////////////////////////////////////////////
52 private CodingStateMachine codingSM;
53 private ProbingState state;
54
55 private SJISContextAnalysis contextAnalyzer;
56 private SJISDistributionAnalysis distributionAnalyzer;
57
58 private byte[] lastChar;
59
60 private static final SMModel smModel = new SJISSMModel();
61
62
63 ////////////////////////////////////////////////////////////////
64 // methods
65 ////////////////////////////////////////////////////////////////
66 public SJISProber()
67 {
68 super();
69 this.codingSM = new CodingStateMachine(smModel);
70 this.contextAnalyzer = new SJISContextAnalysis();
71 this.distributionAnalyzer = new SJISDistributionAnalysis();
72 this.lastChar = new byte[2];
73 reset();
74 }
75
76 @Override
77 public String getCharSetName()
78 {
79 return Constants.CHARSET_SHIFT_JIS;
80 }
81
82 @Override
83 public float getConfidence()
84 {
85 float contextCf = this.contextAnalyzer.getConfidence();
86 float distribCf = this.distributionAnalyzer.getConfidence();
87
88 return Math.max(contextCf, distribCf);
89 }
90
91 @Override
92 public ProbingState getState()
93 {
94 return this.state;
95 }
96
97 @Override
98 public ProbingState handleData(byte[] buf, int offset, int length)
99 {
100 int codingState;
101
102 int maxPos = offset + length;
103 for (int i=offset; i<maxPos; ++i) {
104 codingState = this.codingSM.nextState(buf[i]);
105 if (codingState == SMModel.ERROR) {
106 this.state = ProbingState.NOT_ME;
107 break;
108 }
109 if (codingState == SMModel.ITSME) {
110 this.state = ProbingState.FOUND_IT;
111 break;
112 }
113 if (codingState == SMModel.START) {
114 int charLen = this.codingSM.getCurrentCharLen();
115 if (i == offset) {
116 this.lastChar[1] = buf[offset];
117 this.contextAnalyzer.handleOneChar(this.lastChar, 2-charLen, charLen);
118 this.distributionAnalyzer.handleOneChar(this.lastChar, 0, charLen);
119 } else {
120 this.contextAnalyzer.handleOneChar(buf, i+1-charLen, charLen);
121 this.distributionAnalyzer.handleOneChar(buf, i-1, charLen);
122 }
123 }
124 }
125
126 this.lastChar[0] = buf[maxPos-1];
127
128 if (this.state == ProbingState.DETECTING) {
129 if (this.contextAnalyzer.gotEnoughData() && getConfidence() > SHORTCUT_THRESHOLD) {
130 this.state = ProbingState.FOUND_IT;
131 }
132 }
133
134 return this.state;
135 }
136
137 @Override
138 public void reset()
139 {
140 this.codingSM.reset();
141 this.state = ProbingState.DETECTING;
142 this.contextAnalyzer.reset();
143 this.distributionAnalyzer.reset();
144 java.util.Arrays.fill(this.lastChar, (byte)0);
145 }
146
147 @Override
148 public void setOption()
149 {}
150 }
+0
-194
src/org/mozilla/universalchardet/prober/SingleByteCharsetProber.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei TAKETA <k-tak@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober;
39
40 import org.mozilla.universalchardet.prober.sequence.SequenceModel;
41
42 public class SingleByteCharsetProber extends CharsetProber
43 {
44 ////////////////////////////////////////////////////////////////
45 // constants
46 ////////////////////////////////////////////////////////////////
47 public static final int SAMPLE_SIZE = 64;
48 public static final int SB_ENOUGH_REL_THRESHOLD = 1024;
49 public static final float POSITIVE_SHORTCUT_THRESHOLD = 0.95f;
50 public static final float NEGATIVE_SHORTCUT_THRESHOLD = 0.05f;
51 public static final int SYMBOL_CAT_ORDER = 250;
52 public static final int NUMBER_OF_SEQ_CAT = 4;
53 public static final int POSITIVE_CAT = NUMBER_OF_SEQ_CAT-1;
54 public static final int NEGATIVE_CAT = 0;
55
56
57 ////////////////////////////////////////////////////////////////
58 // fields
59 ////////////////////////////////////////////////////////////////
60 private ProbingState state;
61 private SequenceModel model;
62 private boolean reversed;
63
64 private short lastOrder;
65
66 private int totalSeqs;
67 private int[] seqCounters;
68
69 private int totalChar;
70 private int freqChar;
71
72 private CharsetProber nameProber;
73
74
75 ////////////////////////////////////////////////////////////////
76 // methods
77 ////////////////////////////////////////////////////////////////
78 public SingleByteCharsetProber(SequenceModel model)
79 {
80 super();
81 this.model = model;
82 this.reversed = false;
83 this.nameProber = null;
84 this.seqCounters = new int[NUMBER_OF_SEQ_CAT];
85 reset();
86 }
87
88 public SingleByteCharsetProber(
89 SequenceModel model,
90 boolean reversed,
91 CharsetProber nameProber)
92 {
93 super();
94 this.model = model;
95 this.reversed = reversed;
96 this.nameProber = nameProber;
97 this.seqCounters = new int[NUMBER_OF_SEQ_CAT];
98 reset();
99 }
100
101 boolean keepEnglishLetters()
102 {
103 return this.model.getKeepEnglishLetter();
104 }
105
106 @Override
107 public String getCharSetName()
108 {
109 if (this.nameProber == null) {
110 return this.model.getCharsetName();
111 } else {
112 return this.nameProber.getCharSetName();
113 }
114 }
115
116 @Override
117 public float getConfidence()
118 {
119 if (this.totalSeqs > 0) {
120 float r = 1.0f * this.seqCounters[POSITIVE_CAT] / this.totalSeqs / this.model.getTypicalPositiveRatio();
121 r = r * this.freqChar / this.totalChar;
122 if (r >= 1.0f) {
123 r = 0.99f;
124 }
125 return r;
126 }
127
128 return 0.01f;
129 }
130
131 @Override
132 public ProbingState getState()
133 {
134 return this.state;
135 }
136
137 @Override
138 public ProbingState handleData(byte[] buf, int offset, int length)
139 {
140 short order;
141
142 int maxPos = offset + length;
143 for (int i=offset; i<maxPos; ++i) {
144 order = this.model.getOrder(buf[i]);
145
146 if (order < SYMBOL_CAT_ORDER) {
147 ++this.totalChar;
148 }
149 if (order < SAMPLE_SIZE) {
150 ++this.freqChar;
151 if (this.lastOrder < SAMPLE_SIZE) {
152 ++this.totalSeqs;
153 if (!this.reversed) {
154 ++(this.seqCounters[this.model.getPrecedence(this.lastOrder*SAMPLE_SIZE+order)]);
155 } else {
156 ++(this.seqCounters[this.model.getPrecedence(order*SAMPLE_SIZE+this.lastOrder)]);
157 }
158 }
159 }
160 this.lastOrder = order;
161 }
162
163 if (this.state == ProbingState.DETECTING) {
164 if (this.totalSeqs > SB_ENOUGH_REL_THRESHOLD) {
165 float cf = getConfidence();
166 if (cf > POSITIVE_SHORTCUT_THRESHOLD) {
167 this.state = ProbingState.FOUND_IT;
168 } else if (cf < NEGATIVE_SHORTCUT_THRESHOLD){
169 this.state = ProbingState.NOT_ME;
170 }
171 }
172 }
173
174 return this.state;
175 }
176
177 @Override
178 public void reset()
179 {
180 this.state = ProbingState.DETECTING;
181 this.lastOrder = 255;
182 for (int i=0; i<NUMBER_OF_SEQ_CAT; ++i) {
183 this.seqCounters[i] = 0;
184 }
185 this.totalSeqs = 0;
186 this.totalChar = 0;
187 this.freqChar = 0;
188 }
189
190 @Override
191 public void setOption()
192 {}
193 }
+0
-140
src/org/mozilla/universalchardet/prober/UTF8Prober.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober;
38
39 import org.mozilla.universalchardet.prober.statemachine.CodingStateMachine;
40 import org.mozilla.universalchardet.prober.statemachine.SMModel;
41 import org.mozilla.universalchardet.prober.statemachine.UTF8SMModel;
42 import org.mozilla.universalchardet.Constants;
43
44
45 public class UTF8Prober extends CharsetProber
46 {
47 ////////////////////////////////////////////////////////////////
48 // constants
49 ////////////////////////////////////////////////////////////////
50 public static final float ONE_CHAR_PROB = 0.50f;
51
52
53 ////////////////////////////////////////////////////////////////
54 // fields
55 ////////////////////////////////////////////////////////////////
56 private CodingStateMachine codingSM;
57 private ProbingState state;
58 private int numOfMBChar;
59
60 private static final SMModel smModel = new UTF8SMModel();
61
62
63 ////////////////////////////////////////////////////////////////
64 // methods
65 ////////////////////////////////////////////////////////////////
66 public UTF8Prober()
67 {
68 super();
69 this.numOfMBChar = 0;
70 this.codingSM = new CodingStateMachine(smModel);
71
72 reset();
73 }
74
75 public String getCharSetName()
76 {
77 return Constants.CHARSET_UTF_8;
78 }
79
80 public ProbingState handleData(final byte[] buf, int offset, int length)
81 {
82 int codingState;
83
84 int maxPos = offset + length;
85 for (int i=offset; i<maxPos; ++i) {
86 codingState = this.codingSM.nextState(buf[i]);
87 if (codingState == SMModel.ERROR) {
88 this.state = ProbingState.NOT_ME;
89 break;
90 }
91 if (codingState == SMModel.ITSME) {
92 this.state = ProbingState.FOUND_IT;
93 break;
94 }
95 if (codingState == SMModel.START) {
96 if (this.codingSM.getCurrentCharLen() >= 2) {
97 ++this.numOfMBChar;
98 }
99 }
100 }
101
102 if (this.state == ProbingState.DETECTING) {
103 if (getConfidence() > SHORTCUT_THRESHOLD) {
104 this.state = ProbingState.FOUND_IT;
105 }
106 }
107
108 return this.state;
109 }
110
111 public ProbingState getState()
112 {
113 return this.state;
114 }
115
116 public void reset()
117 {
118 this.codingSM.reset();
119 this.numOfMBChar = 0;
120 this.state = ProbingState.DETECTING;
121 }
122
123 public float getConfidence()
124 {
125 float unlike = 0.99f;
126
127 if (this.numOfMBChar < 6) {
128 for (int i=0; i<this.numOfMBChar; ++i) {
129 unlike *= ONE_CHAR_PROB;
130 }
131 return (1.0f - unlike);
132 } else {
133 return 0.99f;
134 }
135 }
136
137 public void setOption()
138 {}
139 }
+0
-101
src/org/mozilla/universalchardet/prober/contextanalysis/EUCJPContextAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.contextanalysis;
38
39
40 public class EUCJPContextAnalysis extends JapaneseContextAnalysis
41 {
42 ////////////////////////////////////////////////////////////////
43 // constants
44 ////////////////////////////////////////////////////////////////
45 public static final int HIRAGANA_HIGHBYTE = 0xA4;
46 public static final int HIRAGANA_LOWBYTE_BEGIN = 0xA1;
47 public static final int HIRAGANA_LOWBYTE_END = 0xF3;
48 public static final int SINGLE_SHIFT_2 = 0x8E;
49 public static final int SINGLE_SHIFT_3 = 0x8F;
50 public static final int FIRSTPLANE_HIGHBYTE_BEGIN = 0xA1;
51 public static final int FIRSTPLANE_HIGHBYTE_END = 0xFE;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public EUCJPContextAnalysis()
58 {
59 super();
60 }
61
62 @Override
63 protected void getOrder(Order order, final byte[] buf, int offset)
64 {
65 order.order = -1;
66 order.charLength = 1;
67
68 int firstByte = buf[offset] & 0xFF;
69 if (firstByte == SINGLE_SHIFT_2 ||
70 (firstByte >= FIRSTPLANE_HIGHBYTE_BEGIN &&
71 firstByte <= FIRSTPLANE_HIGHBYTE_END)) {
72 order.charLength = 2;
73 } else if (firstByte == SINGLE_SHIFT_3) {
74 order.charLength = 3;
75 }
76
77 if (firstByte == HIRAGANA_HIGHBYTE) {
78 int secondByte = buf[offset+1] & 0xFF;
79 if (secondByte >= HIRAGANA_LOWBYTE_BEGIN &&
80 secondByte <= HIRAGANA_LOWBYTE_END) {
81 order.order = (secondByte - HIRAGANA_LOWBYTE_BEGIN);
82 }
83 }
84 }
85
86 @Override
87 protected int getOrder(final byte[] buf, int offset)
88 {
89 int highbyte = buf[offset] & 0xFF;
90 if (highbyte == HIRAGANA_HIGHBYTE) {
91 int lowbyte = buf[offset+1] & 0xFF;
92 if (lowbyte >= HIRAGANA_LOWBYTE_BEGIN &&
93 lowbyte <= HIRAGANA_LOWBYTE_END) {
94 return (lowbyte - HIRAGANA_LOWBYTE_BEGIN);
95 }
96 }
97
98 return -1;
99 }
100 }
+0
-262
src/org/mozilla/universalchardet/prober/contextanalysis/JapaneseContextAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.contextanalysis;
38
39 public abstract class JapaneseContextAnalysis
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final int NUM_OF_CATEGORY = 6;
45 public static final int ENOUGH_REL_THRESHOLD = 100;
46 public static final int MAX_REL_THRESHOLD = 1000;
47 public static final int MINIMUM_DATA_THRESHOLD = 4;
48 public static final float DONT_KNOW = -1f;
49
50
51 ////////////////////////////////////////////////////////////////
52 // inner types
53 ////////////////////////////////////////////////////////////////
54 protected class Order
55 {
56 public int order;
57 public int charLength;
58
59 public Order()
60 {
61 this.order = -1;
62 this.charLength = 0;
63 }
64 }
65
66 ////////////////////////////////////////////////////////////////
67 // fields
68 ////////////////////////////////////////////////////////////////
69 private int[] relSample = new int[NUM_OF_CATEGORY];
70 private int totalRel;
71 private int lastCharOrder;
72 private int needToSkipCharNum;
73 private boolean done;
74
75 private Order tmpOrder;
76
77
78 ////////////////////////////////////////////////////////////////
79 // methods
80 ////////////////////////////////////////////////////////////////
81 public JapaneseContextAnalysis()
82 {
83 tmpOrder = new Order();
84 reset();
85 }
86
87 public void handleData(final byte[] buf, int offset, int length)
88 {
89 if (this.done) {
90 return;
91 }
92
93 // The buffer we got is byte oriented, and a character may span in more than one
94 // buffers. In case the last one or two byte in last buffer is not complete, we
95 // record how many byte needed to complete that character and skip these bytes here.
96 // We can choose to record those bytes as well and analyse the character once it
97 // is complete, but since a character will not make much difference, by simply skipping
98 // this character will simply our logic and improve performance.
99 int maxPos = offset + length;
100
101 for (int i=this.needToSkipCharNum+offset; i<maxPos; ) {
102 getOrder(this.tmpOrder, buf, i);
103 i += this.tmpOrder.charLength;
104
105 if (i > maxPos) {
106 this.needToSkipCharNum = i - maxPos;
107 this.lastCharOrder = -1;
108 } else {
109 if (this.tmpOrder.order != -1 && this.lastCharOrder != -1) {
110 ++this.totalRel;
111 if (this.totalRel > MAX_REL_THRESHOLD) {
112 this.done = true;
113 break;
114 }
115 ++this.relSample[jp2CharContext[this.lastCharOrder][this.tmpOrder.order]];
116 }
117 }
118 }
119 }
120
121 public void handleOneChar(final byte[] buf, int offset, int charLength)
122 {
123 if (this.totalRel > MAX_REL_THRESHOLD) {
124 this.done = true;
125 }
126 if (this.done) {
127 return;
128 }
129
130
131 int orderNum = -1;
132 if (charLength == 2) {
133 orderNum = getOrder(buf, offset);
134 }
135 if (orderNum != -1 && this.lastCharOrder != -1) {
136 ++this.totalRel;
137 ++this.relSample[jp2CharContext[this.lastCharOrder][orderNum]];
138 }
139 this.lastCharOrder = orderNum;
140 }
141
142 public float getConfidence()
143 {
144 if (this.totalRel > MINIMUM_DATA_THRESHOLD) {
145 return ((float)(this.totalRel - this.relSample[0])) / this.totalRel;
146 } else {
147 return DONT_KNOW;
148 }
149 }
150
151 public void reset()
152 {
153 this.totalRel = 0;
154 for (int i=0; i<NUM_OF_CATEGORY; ++i) {
155 this.relSample[i] = 0;
156 }
157 this.needToSkipCharNum = 0;
158 this.lastCharOrder = -1;
159 this.done = false;
160 }
161
162 public void setOption()
163 {}
164
165 public boolean gotEnoughData()
166 {
167 return (this.totalRel > ENOUGH_REL_THRESHOLD);
168 }
169
170 protected abstract void getOrder(Order order, final byte[] buf, int offset);
171 protected abstract int getOrder(final byte[] buf, int offset);
172
173 ////////////////////////////////////////////////////////////////
174 // constants continued
175 ////////////////////////////////////////////////////////////////
176 protected static final byte[][] jp2CharContext = new byte[][] {
177 { 0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,},
178 { 2,4,0,4,0,3,0,4,0,3,4,4,4,2,4,3,3,4,3,2,3,3,4,2,3,3,3,2,4,1,4,3,3,1,5,4,3,4,3,4,3,5,3,0,3,5,4,2,0,3,1,0,3,3,0,3,3,0,1,1,0,4,3,0,3,3,0,4,0,2,0,3,5,5,5,5,4,0,4,1,0,3,4,},
179 { 0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,},
180 { 0,4,0,5,0,5,0,4,0,4,5,4,4,3,5,3,5,1,5,3,4,3,4,4,3,4,3,3,4,3,5,4,4,3,5,5,3,5,5,5,3,5,5,3,4,5,5,3,1,3,2,0,3,4,0,4,2,0,4,2,1,5,3,2,3,5,0,4,0,2,0,5,4,4,5,4,5,0,4,0,0,4,4,},
181 { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,},
182 { 0,3,0,4,0,3,0,3,0,4,5,4,3,3,3,3,4,3,5,4,4,3,5,4,4,3,4,3,4,4,4,4,5,3,4,4,3,4,5,5,4,5,5,1,4,5,4,3,0,3,3,1,3,3,0,4,4,0,3,3,1,5,3,3,3,5,0,4,0,3,0,4,4,3,4,3,3,0,4,1,1,3,4,},
183 { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,},
184 { 0,4,0,3,0,3,0,4,0,3,4,4,3,2,2,1,2,1,3,1,3,3,3,3,3,4,3,1,3,3,5,3,3,0,4,3,0,5,4,3,3,5,4,4,3,4,4,5,0,1,2,0,1,2,0,2,2,0,1,0,0,5,2,2,1,4,0,3,0,1,0,4,4,3,5,4,3,0,2,1,0,4,3,},
185 { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,},
186 { 0,3,0,5,0,4,0,2,1,4,4,2,4,1,4,2,4,2,4,3,3,3,4,3,3,3,3,1,4,2,3,3,3,1,4,4,1,1,1,4,3,3,2,0,2,4,3,2,0,3,3,0,3,1,1,0,0,0,3,3,0,4,2,2,3,4,0,4,0,3,0,4,4,5,3,4,4,0,3,0,0,1,4,},
187 { 1,4,0,4,0,4,0,4,0,3,5,4,4,3,4,3,5,4,3,3,4,3,5,4,4,4,4,3,4,2,4,3,3,1,5,4,3,2,4,5,4,5,5,4,4,5,4,4,0,3,2,2,3,3,0,4,3,1,3,2,1,4,3,3,4,5,0,3,0,2,0,4,5,5,4,5,4,0,4,0,0,5,4,},
188 { 0,5,0,5,0,4,0,3,0,4,4,3,4,3,3,3,4,0,4,4,4,3,4,3,4,3,3,1,4,2,4,3,4,0,5,4,1,4,5,4,4,5,3,2,4,3,4,3,2,4,1,3,3,3,2,3,2,0,4,3,3,4,3,3,3,4,0,4,0,3,0,4,5,4,4,4,3,0,4,1,0,1,3,},
189 { 0,3,1,4,0,3,0,2,0,3,4,4,3,1,4,2,3,3,4,3,4,3,4,3,4,4,3,2,3,1,5,4,4,1,4,4,3,5,4,4,3,5,5,4,3,4,4,3,1,2,3,1,2,2,0,3,2,0,3,1,0,5,3,3,3,4,3,3,3,3,4,4,4,4,5,4,2,0,3,3,2,4,3,},
190 { 0,2,0,3,0,1,0,1,0,0,3,2,0,0,2,0,1,0,2,1,3,3,3,1,2,3,1,0,1,0,4,2,1,1,3,3,0,4,3,3,1,4,3,3,0,3,3,2,0,0,0,0,1,0,0,2,0,0,0,0,0,4,1,0,2,3,2,2,2,1,3,3,3,4,4,3,2,0,3,1,0,3,3,},
191 { 0,4,0,4,0,3,0,3,0,4,4,4,3,3,3,3,3,3,4,3,4,2,4,3,4,3,3,2,4,3,4,5,4,1,4,5,3,5,4,5,3,5,4,0,3,5,5,3,1,3,3,2,2,3,0,3,4,1,3,3,2,4,3,3,3,4,0,4,0,3,0,4,5,4,4,5,3,0,4,1,0,3,4,},
192 { 0,2,0,3,0,3,0,0,0,2,2,2,1,0,1,0,0,0,3,0,3,0,3,0,1,3,1,0,3,1,3,3,3,1,3,3,3,0,1,3,1,3,4,0,0,3,1,1,0,3,2,0,0,0,0,1,3,0,1,0,0,3,3,2,0,3,0,0,0,0,0,3,4,3,4,3,3,0,3,0,0,2,3,},
193 { 2,3,0,3,0,2,0,1,0,3,3,4,3,1,3,1,1,1,3,1,4,3,4,3,3,3,0,0,3,1,5,4,3,1,4,3,2,5,5,4,4,4,4,3,3,4,4,4,0,2,1,1,3,2,0,1,2,0,0,1,0,4,1,3,3,3,0,3,0,1,0,4,4,4,5,5,3,0,2,0,0,4,4,},
194 { 0,2,0,1,0,3,1,3,0,2,3,3,3,0,3,1,0,0,3,0,3,2,3,1,3,2,1,1,0,0,4,2,1,0,2,3,1,4,3,2,0,4,4,3,1,3,1,3,0,1,0,0,1,0,0,0,1,0,0,0,0,4,1,1,1,2,0,3,0,0,0,3,4,2,4,3,2,0,1,0,0,3,3,},
195 { 0,1,0,4,0,5,0,4,0,2,4,4,2,3,3,2,3,3,5,3,3,3,4,3,4,2,3,0,4,3,3,3,4,1,4,3,2,1,5,5,3,4,5,1,3,5,4,2,0,3,3,0,1,3,0,4,2,0,1,3,1,4,3,3,3,3,0,3,0,1,0,3,4,4,4,5,5,0,3,0,1,4,5,},
196 { 0,2,0,3,0,3,0,0,0,2,3,1,3,0,4,0,1,1,3,0,3,4,3,2,3,1,0,3,3,2,3,1,3,0,2,3,0,2,1,4,1,2,2,0,0,3,3,0,0,2,0,0,0,1,0,0,0,0,2,2,0,3,2,1,3,3,0,2,0,2,0,0,3,3,1,2,4,0,3,0,2,2,3,},
197 { 2,4,0,5,0,4,0,4,0,2,4,4,4,3,4,3,3,3,1,2,4,3,4,3,4,4,5,0,3,3,3,3,2,0,4,3,1,4,3,4,1,4,4,3,3,4,4,3,1,2,3,0,4,2,0,4,1,0,3,3,0,4,3,3,3,4,0,4,0,2,0,3,5,3,4,5,2,0,3,0,0,4,5,},
198 { 0,3,0,4,0,1,0,1,0,1,3,2,2,1,3,0,3,0,2,0,2,0,3,0,2,0,0,0,1,0,1,1,0,0,3,1,0,0,0,4,0,3,1,0,2,1,3,0,0,0,0,0,0,3,0,0,0,0,0,0,0,4,2,2,3,1,0,3,0,0,0,1,4,4,4,3,0,0,4,0,0,1,4,},
199 { 1,4,1,5,0,3,0,3,0,4,5,4,4,3,5,3,3,4,4,3,4,1,3,3,3,3,2,1,4,1,5,4,3,1,4,4,3,5,4,4,3,5,4,3,3,4,4,4,0,3,3,1,2,3,0,3,1,0,3,3,0,5,4,4,4,4,4,4,3,3,5,4,4,3,3,5,4,0,3,2,0,4,4,},
200 { 0,2,0,3,0,1,0,0,0,1,3,3,3,2,4,1,3,0,3,1,3,0,2,2,1,1,0,0,2,0,4,3,1,0,4,3,0,4,4,4,1,4,3,1,1,3,3,1,0,2,0,0,1,3,0,0,0,0,2,0,0,4,3,2,4,3,5,4,3,3,3,4,3,3,4,3,3,0,2,1,0,3,3,},
201 { 0,2,0,4,0,3,0,2,0,2,5,5,3,4,4,4,4,1,4,3,3,0,4,3,4,3,1,3,3,2,4,3,0,3,4,3,0,3,4,4,2,4,4,0,4,5,3,3,2,2,1,1,1,2,0,1,5,0,3,3,2,4,3,3,3,4,0,3,0,2,0,4,4,3,5,5,0,0,3,0,2,3,3,},
202 { 0,3,0,4,0,3,0,1,0,3,4,3,3,1,3,3,3,0,3,1,3,0,4,3,3,1,1,0,3,0,3,3,0,0,4,4,0,1,5,4,3,3,5,0,3,3,4,3,0,2,0,1,1,1,0,1,3,0,1,2,1,3,3,2,3,3,0,3,0,1,0,1,3,3,4,4,1,0,1,2,2,1,3,},
203 { 0,1,0,4,0,4,0,3,0,1,3,3,3,2,3,1,1,0,3,0,3,3,4,3,2,4,2,0,1,0,4,3,2,0,4,3,0,5,3,3,2,4,4,4,3,3,3,4,0,1,3,0,0,1,0,0,1,0,0,0,0,4,2,3,3,3,0,3,0,0,0,4,4,4,5,3,2,0,3,3,0,3,5,},
204 { 0,2,0,3,0,0,0,3,0,1,3,0,2,0,0,0,1,0,3,1,1,3,3,0,0,3,0,0,3,0,2,3,1,0,3,1,0,3,3,2,0,4,2,2,0,2,0,0,0,4,0,0,0,0,0,0,0,0,0,0,0,2,1,2,0,1,0,1,0,0,0,1,3,1,2,0,0,0,1,0,0,1,4,},
205 { 0,3,0,3,0,5,0,1,0,2,4,3,1,3,3,2,1,1,5,2,1,0,5,1,2,0,0,0,3,3,2,2,3,2,4,3,0,0,3,3,1,3,3,0,2,5,3,4,0,3,3,0,1,2,0,2,2,0,3,2,0,2,2,3,3,3,0,2,0,1,0,3,4,4,2,5,4,0,3,0,0,3,5,},
206 { 0,3,0,3,0,3,0,1,0,3,3,3,3,0,3,0,2,0,2,1,1,0,2,0,1,0,0,0,2,1,0,0,1,0,3,2,0,0,3,3,1,2,3,1,0,3,3,0,0,1,0,0,0,0,0,2,0,0,0,0,0,2,3,1,2,3,0,3,0,1,0,3,2,1,0,4,3,0,1,1,0,3,3,},
207 { 0,4,0,5,0,3,0,3,0,4,5,5,4,3,5,3,4,3,5,3,3,2,5,3,4,4,4,3,4,3,4,5,5,3,4,4,3,4,4,5,4,4,4,3,4,5,5,4,2,3,4,2,3,4,0,3,3,1,4,3,2,4,3,3,5,5,0,3,0,3,0,5,5,5,5,4,4,0,4,0,1,4,4,},
208 { 0,4,0,4,0,3,0,3,0,3,5,4,4,2,3,2,5,1,3,2,5,1,4,2,3,2,3,3,4,3,3,3,3,2,5,4,1,3,3,5,3,4,4,0,4,4,3,1,1,3,1,0,2,3,0,2,3,0,3,0,0,4,3,1,3,4,0,3,0,2,0,4,4,4,3,4,5,0,4,0,0,3,4,},
209 { 0,3,0,3,0,3,1,2,0,3,4,4,3,3,3,0,2,2,4,3,3,1,3,3,3,1,1,0,3,1,4,3,2,3,4,4,2,4,4,4,3,4,4,3,2,4,4,3,1,3,3,1,3,3,0,4,1,0,2,2,1,4,3,2,3,3,5,4,3,3,5,4,4,3,3,0,4,0,3,2,2,4,4,},
210 { 0,2,0,1,0,0,0,0,0,1,2,1,3,0,0,0,0,0,2,0,1,2,1,0,0,1,0,0,0,0,3,0,0,1,0,1,1,3,1,0,0,0,1,1,0,1,1,0,0,0,0,0,2,0,0,0,0,0,0,0,0,1,1,2,2,0,3,4,0,0,0,1,1,0,0,1,0,0,0,0,0,1,1,},
211 { 0,1,0,0,0,1,0,0,0,0,4,0,4,1,4,0,3,0,4,0,3,0,4,0,3,0,3,0,4,1,5,1,4,0,0,3,0,5,0,5,2,0,1,0,0,0,2,1,4,0,1,3,0,0,3,0,0,3,1,1,4,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,},
212 { 1,4,0,5,0,3,0,2,0,3,5,4,4,3,4,3,5,3,4,3,3,0,4,3,3,3,3,3,3,2,4,4,3,1,3,4,4,5,4,4,3,4,4,1,3,5,4,3,3,3,1,2,2,3,3,1,3,1,3,3,3,5,3,3,4,5,0,3,0,3,0,3,4,3,4,4,3,0,3,0,2,4,3,},
213 { 0,1,0,4,0,0,0,0,0,1,4,0,4,1,4,2,4,0,3,0,1,0,1,0,0,0,0,0,2,0,3,1,1,1,0,3,0,0,0,1,2,1,0,0,1,1,1,1,0,1,0,0,0,1,0,0,3,0,0,0,0,3,2,0,2,2,0,1,0,0,0,2,3,2,3,3,0,0,0,0,2,1,0,},
214 { 0,5,1,5,0,3,0,3,0,5,4,4,5,1,5,3,3,0,4,3,4,3,5,3,4,3,3,2,4,3,4,3,3,0,3,3,1,4,4,3,4,4,4,3,4,5,5,3,2,3,1,1,3,3,1,3,1,1,3,3,2,4,5,3,3,5,0,4,0,3,0,4,4,3,5,3,3,0,3,4,0,4,3,},
215 { 0,5,0,5,0,3,0,2,0,4,4,3,5,2,4,3,3,3,4,4,4,3,5,3,5,3,3,1,4,0,4,3,3,0,3,3,0,4,4,4,4,5,4,3,3,5,5,3,2,3,1,2,3,2,0,1,0,0,3,2,2,4,4,3,1,5,0,4,0,3,0,4,3,1,3,2,1,0,3,3,0,3,3,},
216 { 0,4,0,5,0,5,0,4,0,4,5,5,5,3,4,3,3,2,5,4,4,3,5,3,5,3,4,0,4,3,4,4,3,2,4,4,3,4,5,4,4,5,5,0,3,5,5,4,1,3,3,2,3,3,1,3,1,0,4,3,1,4,4,3,4,5,0,4,0,2,0,4,3,4,4,3,3,0,4,0,0,5,5,},
217 { 0,4,0,4,0,5,0,1,1,3,3,4,4,3,4,1,3,0,5,1,3,0,3,1,3,1,1,0,3,0,3,3,4,0,4,3,0,4,4,4,3,4,4,0,3,5,4,1,0,3,0,0,2,3,0,3,1,0,3,1,0,3,2,1,3,5,0,3,0,1,0,3,2,3,3,4,4,0,2,2,0,4,4,},
218 { 2,4,0,5,0,4,0,3,0,4,5,5,4,3,5,3,5,3,5,3,5,2,5,3,4,3,3,4,3,4,5,3,2,1,5,4,3,2,3,4,5,3,4,1,2,5,4,3,0,3,3,0,3,2,0,2,3,0,4,1,0,3,4,3,3,5,0,3,0,1,0,4,5,5,5,4,3,0,4,2,0,3,5,},
219 { 0,5,0,4,0,4,0,2,0,5,4,3,4,3,4,3,3,3,4,3,4,2,5,3,5,3,4,1,4,3,4,4,4,0,3,5,0,4,4,4,4,5,3,1,3,4,5,3,3,3,3,3,3,3,0,2,2,0,3,3,2,4,3,3,3,5,3,4,1,3,3,5,3,2,0,0,0,0,4,3,1,3,3,},
220 { 0,1,0,3,0,3,0,1,0,1,3,3,3,2,3,3,3,0,3,0,0,0,3,1,3,0,0,0,2,2,2,3,0,0,3,2,0,1,2,4,1,3,3,0,0,3,3,3,0,1,0,0,2,1,0,0,3,0,3,1,0,3,0,0,1,3,0,2,0,1,0,3,3,1,3,3,0,0,1,1,0,3,3,},
221 { 0,2,0,3,0,2,1,4,0,2,2,3,1,1,3,1,1,0,2,0,3,1,2,3,1,3,0,0,1,0,4,3,2,3,3,3,1,4,2,3,3,3,3,1,0,3,1,4,0,1,1,0,1,2,0,1,1,0,1,1,0,3,1,3,2,2,0,1,0,0,0,2,3,3,3,1,0,0,0,0,0,2,3,},
222 { 0,5,0,4,0,5,0,2,0,4,5,5,3,3,4,3,3,1,5,4,4,2,4,4,4,3,4,2,4,3,5,5,4,3,3,4,3,3,5,5,4,5,5,1,3,4,5,3,1,4,3,1,3,3,0,3,3,1,4,3,1,4,5,3,3,5,0,4,0,3,0,5,3,3,1,4,3,0,4,0,1,5,3,},
223 { 0,5,0,5,0,4,0,2,0,4,4,3,4,3,3,3,3,3,5,4,4,4,4,4,4,5,3,3,5,2,4,4,4,3,4,4,3,3,4,4,5,5,3,3,4,3,4,3,3,4,3,3,3,3,1,2,2,1,4,3,3,5,4,4,3,4,0,4,0,3,0,4,4,4,4,4,1,0,4,2,0,2,4,},
224 { 0,4,0,4,0,3,0,1,0,3,5,2,3,0,3,0,2,1,4,2,3,3,4,1,4,3,3,2,4,1,3,3,3,0,3,3,0,0,3,3,3,5,3,3,3,3,3,2,0,2,0,0,2,0,0,2,0,0,1,0,0,3,1,2,2,3,0,3,0,2,0,4,4,3,3,4,1,0,3,0,0,2,4,},
225 { 0,0,0,4,0,0,0,0,0,0,1,0,1,0,2,0,0,0,0,0,1,0,2,0,1,0,0,0,0,0,3,1,3,0,3,2,0,0,0,1,0,3,2,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,4,0,2,0,0,0,0,0,0,2,},
226 { 0,2,1,3,0,2,0,2,0,3,3,3,3,1,3,1,3,3,3,3,3,3,4,2,2,1,2,1,4,0,4,3,1,3,3,3,2,4,3,5,4,3,3,3,3,3,3,3,0,1,3,0,2,0,0,1,0,0,1,0,0,4,2,0,2,3,0,3,3,0,3,3,4,2,3,1,4,0,1,2,0,2,3,},
227 { 0,3,0,3,0,1,0,3,0,2,3,3,3,0,3,1,2,0,3,3,2,3,3,2,3,2,3,1,3,0,4,3,2,0,3,3,1,4,3,3,2,3,4,3,1,3,3,1,1,0,1,1,0,1,0,1,0,1,0,0,0,4,1,1,0,3,0,3,1,0,2,3,3,3,3,3,1,0,0,2,0,3,3,},
228 { 0,0,0,0,0,0,0,0,0,0,3,0,2,0,3,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,3,0,3,0,3,1,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,2,0,2,3,0,0,0,0,0,0,0,0,3,},
229 { 0,2,0,3,1,3,0,3,0,2,3,3,3,1,3,1,3,1,3,1,3,3,3,1,3,0,2,3,1,1,4,3,3,2,3,3,1,2,2,4,1,3,3,0,1,4,2,3,0,1,3,0,3,0,0,1,3,0,2,0,0,3,3,2,1,3,0,3,0,2,0,3,4,4,4,3,1,0,3,0,0,3,3,},
230 { 0,2,0,1,0,2,0,0,0,1,3,2,2,1,3,0,1,1,3,0,3,2,3,1,2,0,2,0,1,1,3,3,3,0,3,3,1,1,2,3,2,3,3,1,2,3,2,0,0,1,0,0,0,0,0,0,3,0,1,0,0,2,1,2,1,3,0,3,0,0,0,3,4,4,4,3,2,0,2,0,0,2,4,},
231 { 0,0,0,1,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,1,1,1,0,0,0,0,0,0,0,0,0,2,2,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,3,1,0,0,0,0,0,0,0,3,},
232 { 0,3,0,3,0,2,0,3,0,3,3,3,2,3,2,2,2,0,3,1,3,3,3,2,3,3,0,0,3,0,3,2,2,0,2,3,1,4,3,4,3,3,2,3,1,5,4,4,0,3,1,2,1,3,0,3,1,1,2,0,2,3,1,3,1,3,0,3,0,1,0,3,3,4,4,2,1,0,2,1,0,2,4,},
233 { 0,1,0,3,0,1,0,2,0,1,4,2,5,1,4,0,2,0,2,1,3,1,4,0,2,1,0,0,2,1,4,1,1,0,3,3,0,5,1,3,2,3,3,1,0,3,2,3,0,1,0,0,0,0,0,0,1,0,0,0,0,4,0,1,0,3,0,2,0,1,0,3,3,3,4,3,3,0,0,0,0,2,3,},
234 { 0,0,0,1,0,0,0,0,0,0,2,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,1,0,0,1,0,0,0,0,0,3,},
235 { 0,1,0,3,0,4,0,3,0,2,4,3,1,0,3,2,2,1,3,1,2,2,3,1,1,1,2,1,3,0,1,2,0,1,3,2,1,3,0,5,5,1,0,0,1,3,2,1,0,3,0,0,1,0,0,0,0,0,3,4,0,1,1,1,3,2,0,2,0,1,0,2,3,3,1,2,3,0,1,0,1,0,4,},
236 { 0,0,0,1,0,3,0,3,0,2,2,1,0,0,4,0,3,0,3,1,3,0,3,0,3,0,1,0,3,0,3,1,3,0,3,3,0,0,1,2,1,1,1,0,1,2,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,2,2,1,2,0,0,2,0,0,0,0,2,3,3,3,3,0,0,0,0,1,4,},
237 { 0,0,0,3,0,3,0,0,0,0,3,1,1,0,3,0,1,0,2,0,1,0,0,0,0,0,0,0,1,0,3,0,2,0,2,3,0,0,2,2,3,1,2,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,2,0,0,0,0,2,3,},
238 { 2,4,0,5,0,5,0,4,0,3,4,3,3,3,4,3,3,3,4,3,4,4,5,4,5,5,5,2,3,0,5,5,4,1,5,4,3,1,5,4,3,4,4,3,3,4,3,3,0,3,2,0,2,3,0,3,0,0,3,3,0,5,3,2,3,3,0,3,0,3,0,3,4,5,4,5,3,0,4,3,0,3,4,},
239 { 0,3,0,3,0,3,0,3,0,3,3,4,3,2,3,2,3,0,4,3,3,3,3,3,3,3,3,0,3,2,4,3,3,1,3,4,3,4,4,4,3,4,4,3,2,4,4,1,0,2,0,0,1,1,0,2,0,0,3,1,0,5,3,2,1,3,0,3,0,1,2,4,3,2,4,3,3,0,3,2,0,4,4,},
240 { 0,3,0,3,0,1,0,0,0,1,4,3,3,2,3,1,3,1,4,2,3,2,4,2,3,4,3,0,2,2,3,3,3,0,3,3,3,0,3,4,1,3,3,0,3,4,3,3,0,1,1,0,1,0,0,0,4,0,3,0,0,3,1,2,1,3,0,4,0,1,0,4,3,3,4,3,3,0,2,0,0,3,3,},
241 { 0,3,0,4,0,1,0,3,0,3,4,3,3,0,3,3,3,1,3,1,3,3,4,3,3,3,0,0,3,1,5,3,3,1,3,3,2,5,4,3,3,4,5,3,2,5,3,4,0,1,0,0,0,0,0,2,0,0,1,1,0,4,2,2,1,3,0,3,0,2,0,4,4,3,5,3,2,0,1,1,0,3,4,},
242 { 0,5,0,4,0,5,0,2,0,4,4,3,3,2,3,3,3,1,4,3,4,1,5,3,4,3,4,0,4,2,4,3,4,1,5,4,0,4,4,4,4,5,4,1,3,5,4,2,1,4,1,1,3,2,0,3,1,0,3,2,1,4,3,3,3,4,0,4,0,3,0,4,4,4,3,3,3,0,4,2,0,3,4,},
243 { 1,4,0,4,0,3,0,1,0,3,3,3,1,1,3,3,2,2,3,3,1,0,3,2,2,1,2,0,3,1,2,1,2,0,3,2,0,2,2,3,3,4,3,0,3,3,1,2,0,1,1,3,1,2,0,0,3,0,1,1,0,3,2,2,3,3,0,3,0,0,0,2,3,3,4,3,3,0,1,0,0,1,4,},
244 { 0,4,0,4,0,4,0,0,0,3,4,4,3,1,4,2,3,2,3,3,3,1,4,3,4,0,3,0,4,2,3,3,2,2,5,4,2,1,3,4,3,4,3,1,3,3,4,2,0,2,1,0,3,3,0,0,2,0,3,1,0,4,4,3,4,3,0,4,0,1,0,2,4,4,4,4,4,0,3,2,0,3,3,},
245 { 0,0,0,1,0,4,0,0,0,0,0,0,1,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,3,2,0,0,1,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,2,},
246 { 0,2,0,3,0,4,0,4,0,1,3,3,3,0,4,0,2,1,2,1,1,1,2,0,3,1,1,0,1,0,3,1,0,0,3,3,2,0,1,1,0,0,0,0,0,1,0,2,0,2,2,0,3,1,0,0,1,0,1,1,0,1,2,0,3,0,0,0,0,1,0,0,3,3,4,3,1,0,1,0,3,0,2,},
247 { 0,0,0,3,0,5,0,0,0,0,1,0,2,0,3,1,0,1,3,0,0,0,2,0,0,0,1,0,0,0,1,1,0,0,4,0,0,0,2,3,0,1,4,1,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,1,0,0,0,0,0,0,0,2,0,0,3,0,0,0,0,0,3,},
248 { 0,2,0,5,0,5,0,1,0,2,4,3,3,2,5,1,3,2,3,3,3,0,4,1,2,0,3,0,4,0,2,2,1,1,5,3,0,0,1,4,2,3,2,0,3,3,3,2,0,2,4,1,1,2,0,1,1,0,3,1,0,1,3,1,2,3,0,2,0,0,0,1,3,5,4,4,4,0,3,0,0,1,3,},
249 { 0,4,0,5,0,4,0,4,0,4,5,4,3,3,4,3,3,3,4,3,4,4,5,3,4,5,4,2,4,2,3,4,3,1,4,4,1,3,5,4,4,5,5,4,4,5,5,5,2,3,3,1,4,3,1,3,3,0,3,3,1,4,3,4,4,4,0,3,0,4,0,3,3,4,4,5,0,0,4,3,0,4,5,},
250 { 0,4,0,4,0,3,0,3,0,3,4,4,4,3,3,2,4,3,4,3,4,3,5,3,4,3,2,1,4,2,4,4,3,1,3,4,2,4,5,5,3,4,5,4,1,5,4,3,0,3,2,2,3,2,1,3,1,0,3,3,3,5,3,3,3,5,4,4,2,3,3,4,3,3,3,2,1,0,3,2,1,4,3,},
251 { 0,4,0,5,0,4,0,3,0,3,5,5,3,2,4,3,4,0,5,4,4,1,4,4,4,3,3,3,4,3,5,5,2,3,3,4,1,2,5,5,3,5,5,2,3,5,5,4,0,3,2,0,3,3,1,1,5,1,4,1,0,4,3,2,3,5,0,4,0,3,0,5,4,3,4,3,0,0,4,1,0,4,4,},
252 { 1,3,0,4,0,2,0,2,0,2,5,5,3,3,3,3,3,0,4,2,3,4,4,4,3,4,0,0,3,4,5,4,3,3,3,3,2,5,5,4,5,5,5,4,3,5,5,5,1,3,1,0,1,0,0,3,2,0,4,2,0,5,2,3,2,4,1,3,0,3,0,4,5,4,5,4,3,0,4,2,0,5,4,},
253 { 0,3,0,4,0,5,0,3,0,3,4,4,3,2,3,2,3,3,3,3,3,2,4,3,3,2,2,0,3,3,3,3,3,1,3,3,3,0,4,4,3,4,4,1,1,4,4,2,0,3,1,0,1,1,0,4,1,0,2,3,1,3,3,1,3,4,0,3,0,1,0,3,1,3,0,0,1,0,2,0,0,4,4,},
254 { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,},
255 { 0,3,0,3,0,2,0,3,0,1,5,4,3,3,3,1,4,2,1,2,3,4,4,2,4,4,5,0,3,1,4,3,4,0,4,3,3,3,2,3,2,5,3,4,3,2,2,3,0,0,3,0,2,1,0,1,2,0,0,0,0,2,1,1,3,1,0,2,0,4,0,3,4,4,4,5,2,0,2,0,0,1,3,},
256 { 0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,1,1,0,0,1,1,0,0,0,4,2,1,1,0,1,0,3,2,0,0,3,1,1,1,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,1,0,0,0,2,0,0,0,1,4,0,4,2,1,0,0,0,0,0,1,},
257 { 0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,3,1,0,0,0,2,0,2,1,0,0,1,2,1,0,1,1,0,0,3,0,0,0,0,0,0,0,0,0,0,0,1,3,1,0,0,0,0,0,1,0,0,2,1,0,0,0,0,0,0,0,0,2,},
258 { 0,4,0,4,0,4,0,3,0,4,4,3,4,2,4,3,2,0,4,4,4,3,5,3,5,3,3,2,4,2,4,3,4,3,1,4,0,2,3,4,4,4,3,3,3,4,4,4,3,4,1,3,4,3,2,1,2,1,3,3,3,4,4,3,3,5,0,4,0,3,0,4,3,3,3,2,1,0,3,0,0,3,3,},
259 { 0,4,0,3,0,3,0,3,0,3,5,5,3,3,3,3,4,3,4,3,3,3,4,4,4,3,3,3,3,4,3,5,3,3,1,3,2,4,5,5,5,5,4,3,4,5,5,3,2,2,3,3,3,3,2,3,3,1,2,3,2,4,3,3,3,4,0,4,0,2,0,4,3,2,2,1,2,0,3,0,0,4,1,},
260 };
261 }
+0
-97
src/org/mozilla/universalchardet/prober/contextanalysis/SJISContextAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.contextanalysis;
38
39 public class SJISContextAnalysis extends JapaneseContextAnalysis
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final int HIRAGANA_HIGHBYTE = 0x82;
45 public static final int HIRAGANA_LOWBYTE_BEGIN = 0x9F;
46 public static final int HIRAGANA_LOWBYTE_END = 0xF1;
47 public static final int HIGHBYTE_BEGIN_1 = 0x81;
48 public static final int HIGHBYTE_END_1 = 0x9F;
49 public static final int HIGHBYTE_BEGIN_2 = 0xE0;
50 public static final int HIGHBYTE_END_2 = 0xEF;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public SJISContextAnalysis()
57 {
58 super();
59 }
60
61 @Override
62 protected void getOrder(Order order, final byte[] buf, int offset)
63 {
64 order.order = -1;
65 order.charLength = 1;
66
67 int highbyte = buf[offset] & 0xFF;
68 if ((highbyte >= HIGHBYTE_BEGIN_1 && highbyte <= HIGHBYTE_END_1) ||
69 (highbyte >= HIGHBYTE_BEGIN_2 && highbyte <= HIGHBYTE_END_2)) {
70 order.charLength = 2;
71 }
72
73 if (highbyte == HIRAGANA_HIGHBYTE) {
74 int lowbyte = buf[offset+1] & 0xFF;
75 if (lowbyte >= HIRAGANA_LOWBYTE_BEGIN &&
76 lowbyte <= HIRAGANA_LOWBYTE_END) {
77 order.order = (lowbyte - HIRAGANA_LOWBYTE_BEGIN);
78 }
79 }
80 }
81
82 @Override
83 protected int getOrder(final byte[] buf, int offset)
84 {
85 int highbyte = buf[offset] & 0xFF;
86 if (highbyte == HIRAGANA_HIGHBYTE) {
87 int lowbyte = buf[offset+1] & 0xFF;
88 if (lowbyte >= HIRAGANA_LOWBYTE_BEGIN &&
89 lowbyte <= HIRAGANA_LOWBYTE_END) {
90 return (lowbyte - HIRAGANA_LOWBYTE_BEGIN);
91 }
92 }
93
94 return -1;
95 }
96 }
+0
-965
src/org/mozilla/universalchardet/prober/distributionanalysis/Big5DistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public class Big5DistributionAnalysis extends CharDistributionAnalysis
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final float BIG5_TYPICAL_DISTRIBUTION_RATIO = 0.75f;
45 public static final int HIGHBYTE_BEGIN = 0xA4;
46 public static final int HIGHBYTE_END = 0xFE;
47 public static final int LOWBYTE_BEGIN_1 = 0x40;
48 public static final int LOWBYTE_END_1 = 0x7E;
49 public static final int LOWBYTE_BEGIN_2 = 0xA1;
50 public static final int LOWBYTE_END_2 = 0xFE;
51
52
53 ////////////////////////////////////////////////////////////////
54 // methods
55 ////////////////////////////////////////////////////////////////
56 public Big5DistributionAnalysis()
57 {
58 super();
59 this.charToFreqOrder = big5CharToFreqOrder;
60 this.typicalDistributionRatio = BIG5_TYPICAL_DISTRIBUTION_RATIO;
61 }
62
63 protected int getOrder(final byte[] buf, int offset)
64 {
65 int highbyte = buf[offset] & 0xFF;
66 if (highbyte >= HIGHBYTE_BEGIN) {
67 int lowbyte = buf[offset+1] & 0xFF;
68 if (lowbyte >= LOWBYTE_BEGIN_2) {
69 return (157 * (highbyte - HIGHBYTE_BEGIN) + lowbyte - LOWBYTE_BEGIN_2 + 63);
70 } else {
71 return (157 * (highbyte - HIGHBYTE_BEGIN) + lowbyte - LOWBYTE_BEGIN_1);
72 }
73 }
74
75 return -1;
76 }
77
78
79 ////////////////////////////////////////////////////////////////
80 // constants continued
81 ////////////////////////////////////////////////////////////////
82 private static final int[] big5CharToFreqOrder = new int[] {
83 1,1801,1506, 255,1431, 198, 9, 82, 6,5008, 177, 202,3681,1256,2821, 110, // 16
84 3814, 33,3274, 261, 76, 44,2114, 16,2946,2187,1176, 659,3971, 26,3451,2653, // 32
85 1198,3972,3350,4202, 410,2215, 302, 590, 361,1964, 8, 204, 58,4510,5009,1932, // 48
86 63,5010,5011, 317,1614, 75, 222, 159,4203,2417,1480,5012,3555,3091, 224,2822, // 64
87 3682, 3, 10,3973,1471, 29,2787,1135,2866,1940, 873, 130,3275,1123, 312,5013, // 80
88 4511,2052, 507, 252, 682,5014, 142,1915, 124, 206,2947, 34,3556,3204, 64, 604, // 96
89 5015,2501,1977,1978, 155,1991, 645, 641,1606,5016,3452, 337, 72, 406,5017, 80, // 112
90 630, 238,3205,1509, 263, 939,1092,2654, 756,1440,1094,3453, 449, 69,2987, 591, // 128
91 179,2096, 471, 115,2035,1844, 60, 50,2988, 134, 806,1869, 734,2036,3454, 180, // 144
92 995,1607, 156, 537,2907, 688,5018, 319,1305, 779,2145, 514,2379, 298,4512, 359, // 160
93 2502, 90,2716,1338, 663, 11, 906,1099,2553, 20,2441, 182, 532,1716,5019, 732, // 176
94 1376,4204,1311,1420,3206, 25,2317,1056, 113, 399, 382,1950, 242,3455,2474, 529, // 192
95 3276, 475,1447,3683,5020, 117, 21, 656, 810,1297,2300,2334,3557,5021, 126,4205, // 208
96 706, 456, 150, 613,4513, 71,1118,2037,4206, 145,3092, 85, 835, 486,2115,1246, // 224
97 1426, 428, 727,1285,1015, 800, 106, 623, 303,1281,5022,2128,2359, 347,3815, 221, // 240
98 3558,3135,5023,1956,1153,4207, 83, 296,1199,3093, 192, 624, 93,5024, 822,1898, // 256
99 2823,3136, 795,2065, 991,1554,1542,1592, 27, 43,2867, 859, 139,1456, 860,4514, // 272
100 437, 712,3974, 164,2397,3137, 695, 211,3037,2097, 195,3975,1608,3559,3560,3684, // 288
101 3976, 234, 811,2989,2098,3977,2233,1441,3561,1615,2380, 668,2077,1638, 305, 228, // 304
102 1664,4515, 467, 415,5025, 262,2099,1593, 239, 108, 300, 200,1033, 512,1247,2078, // 320
103 5026,5027,2176,3207,3685,2682, 593, 845,1062,3277, 88,1723,2038,3978,1951, 212, // 336
104 266, 152, 149, 468,1899,4208,4516, 77, 187,5028,3038, 37, 5,2990,5029,3979, // 352
105 5030,5031, 39,2524,4517,2908,3208,2079, 55, 148, 74,4518, 545, 483,1474,1029, // 368
106 1665, 217,1870,1531,3138,1104,2655,4209, 24, 172,3562, 900,3980,3563,3564,4519, // 384
107 32,1408,2824,1312, 329, 487,2360,2251,2717, 784,2683, 4,3039,3351,1427,1789, // 400
108 188, 109, 499,5032,3686,1717,1790, 888,1217,3040,4520,5033,3565,5034,3352,1520, // 416
109 3687,3981, 196,1034, 775,5035,5036, 929,1816, 249, 439, 38,5037,1063,5038, 794, // 432
110 3982,1435,2301, 46, 178,3278,2066,5039,2381,5040, 214,1709,4521, 804, 35, 707, // 448
111 324,3688,1601,2554, 140, 459,4210,5041,5042,1365, 839, 272, 978,2262,2580,3456, // 464
112 2129,1363,3689,1423, 697, 100,3094, 48, 70,1231, 495,3139,2196,5043,1294,5044, // 480
113 2080, 462, 586,1042,3279, 853, 256, 988, 185,2382,3457,1698, 434,1084,5045,3458, // 496
114 314,2625,2788,4522,2335,2336, 569,2285, 637,1817,2525, 757,1162,1879,1616,3459, // 512
115 287,1577,2116, 768,4523,1671,2868,3566,2526,1321,3816, 909,2418,5046,4211, 933, // 528
116 3817,4212,2053,2361,1222,4524, 765,2419,1322, 786,4525,5047,1920,1462,1677,2909, // 544
117 1699,5048,4526,1424,2442,3140,3690,2600,3353,1775,1941,3460,3983,4213, 309,1369, // 560
118 1130,2825, 364,2234,1653,1299,3984,3567,3985,3986,2656, 525,1085,3041, 902,2001, // 576
119 1475, 964,4527, 421,1845,1415,1057,2286, 940,1364,3141, 376,4528,4529,1381, 7, // 592
120 2527, 983,2383, 336,1710,2684,1846, 321,3461, 559,1131,3042,2752,1809,1132,1313, // 608
121 265,1481,1858,5049, 352,1203,2826,3280, 167,1089, 420,2827, 776, 792,1724,3568, // 624
122 4214,2443,3281,5050,4215,5051, 446, 229, 333,2753, 901,3818,1200,1557,4530,2657, // 640
123 1921, 395,2754,2685,3819,4216,1836, 125, 916,3209,2626,4531,5052,5053,3820,5054, // 656
124 5055,5056,4532,3142,3691,1133,2555,1757,3462,1510,2318,1409,3569,5057,2146, 438, // 672
125 2601,2910,2384,3354,1068, 958,3043, 461, 311,2869,2686,4217,1916,3210,4218,1979, // 688
126 383, 750,2755,2627,4219, 274, 539, 385,1278,1442,5058,1154,1965, 384, 561, 210, // 704
127 98,1295,2556,3570,5059,1711,2420,1482,3463,3987,2911,1257, 129,5060,3821, 642, // 720
128 523,2789,2790,2658,5061, 141,2235,1333, 68, 176, 441, 876, 907,4220, 603,2602, // 736
129 710, 171,3464, 404, 549, 18,3143,2398,1410,3692,1666,5062,3571,4533,2912,4534, // 752
130 5063,2991, 368,5064, 146, 366, 99, 871,3693,1543, 748, 807,1586,1185, 22,2263, // 768
131 379,3822,3211,5065,3212, 505,1942,2628,1992,1382,2319,5066, 380,2362, 218, 702, // 784
132 1818,1248,3465,3044,3572,3355,3282,5067,2992,3694, 930,3283,3823,5068, 59,5069, // 800
133 585, 601,4221, 497,3466,1112,1314,4535,1802,5070,1223,1472,2177,5071, 749,1837, // 816
134 690,1900,3824,1773,3988,1476, 429,1043,1791,2236,2117, 917,4222, 447,1086,1629, // 832
135 5072, 556,5073,5074,2021,1654, 844,1090, 105, 550, 966,1758,2828,1008,1783, 686, // 848
136 1095,5075,2287, 793,1602,5076,3573,2603,4536,4223,2948,2302,4537,3825, 980,2503, // 864
137 544, 353, 527,4538, 908,2687,2913,5077, 381,2629,1943,1348,5078,1341,1252, 560, // 880
138 3095,5079,3467,2870,5080,2054, 973, 886,2081, 143,4539,5081,5082, 157,3989, 496, // 896
139 4224, 57, 840, 540,2039,4540,4541,3468,2118,1445, 970,2264,1748,1966,2082,4225, // 912
140 3144,1234,1776,3284,2829,3695, 773,1206,2130,1066,2040,1326,3990,1738,1725,4226, // 928
141 279,3145, 51,1544,2604, 423,1578,2131,2067, 173,4542,1880,5083,5084,1583, 264, // 944
142 610,3696,4543,2444, 280, 154,5085,5086,5087,1739, 338,1282,3096, 693,2871,1411, // 960
143 1074,3826,2445,5088,4544,5089,5090,1240, 952,2399,5091,2914,1538,2688, 685,1483, // 976
144 4227,2475,1436, 953,4228,2055,4545, 671,2400, 79,4229,2446,3285, 608, 567,2689, // 992
145 3469,4230,4231,1691, 393,1261,1792,2401,5092,4546,5093,5094,5095,5096,1383,1672, // 1008
146 3827,3213,1464, 522,1119, 661,1150, 216, 675,4547,3991,1432,3574, 609,4548,2690, // 1024
147 2402,5097,5098,5099,4232,3045, 0,5100,2476, 315, 231,2447, 301,3356,4549,2385, // 1040
148 5101, 233,4233,3697,1819,4550,4551,5102, 96,1777,1315,2083,5103, 257,5104,1810, // 1056
149 3698,2718,1139,1820,4234,2022,1124,2164,2791,1778,2659,5105,3097, 363,1655,3214, // 1072
150 5106,2993,5107,5108,5109,3992,1567,3993, 718, 103,3215, 849,1443, 341,3357,2949, // 1088
151 1484,5110,1712, 127, 67, 339,4235,2403, 679,1412, 821,5111,5112, 834, 738, 351, // 1104
152 2994,2147, 846, 235,1497,1881, 418,1993,3828,2719, 186,1100,2148,2756,3575,1545, // 1120
153 1355,2950,2872,1377, 583,3994,4236,2581,2995,5113,1298,3699,1078,2557,3700,2363, // 1136
154 78,3829,3830, 267,1289,2100,2002,1594,4237, 348, 369,1274,2197,2178,1838,4552, // 1152
155 1821,2830,3701,2757,2288,2003,4553,2951,2758, 144,3358, 882,4554,3995,2759,3470, // 1168
156 4555,2915,5114,4238,1726, 320,5115,3996,3046, 788,2996,5116,2831,1774,1327,2873, // 1184
157 3997,2832,5117,1306,4556,2004,1700,3831,3576,2364,2660, 787,2023, 506, 824,3702, // 1200
158 534, 323,4557,1044,3359,2024,1901, 946,3471,5118,1779,1500,1678,5119,1882,4558, // 1216
159 165, 243,4559,3703,2528, 123, 683,4239, 764,4560, 36,3998,1793, 589,2916, 816, // 1232
160 626,1667,3047,2237,1639,1555,1622,3832,3999,5120,4000,2874,1370,1228,1933, 891, // 1248
161 2084,2917, 304,4240,5121, 292,2997,2720,3577, 691,2101,4241,1115,4561, 118, 662, // 1264
162 5122, 611,1156, 854,2386,1316,2875, 2, 386, 515,2918,5123,5124,3286, 868,2238, // 1280
163 1486, 855,2661, 785,2216,3048,5125,1040,3216,3578,5126,3146, 448,5127,1525,5128, // 1296
164 2165,4562,5129,3833,5130,4242,2833,3579,3147, 503, 818,4001,3148,1568, 814, 676, // 1312
165 1444, 306,1749,5131,3834,1416,1030, 197,1428, 805,2834,1501,4563,5132,5133,5134, // 1328
166 1994,5135,4564,5136,5137,2198, 13,2792,3704,2998,3149,1229,1917,5138,3835,2132, // 1344
167 5139,4243,4565,2404,3580,5140,2217,1511,1727,1120,5141,5142, 646,3836,2448, 307, // 1360
168 5143,5144,1595,3217,5145,5146,5147,3705,1113,1356,4002,1465,2529,2530,5148, 519, // 1376
169 5149, 128,2133, 92,2289,1980,5150,4003,1512, 342,3150,2199,5151,2793,2218,1981, // 1392
170 3360,4244, 290,1656,1317, 789, 827,2365,5152,3837,4566, 562, 581,4004,5153, 401, // 1408
171 4567,2252, 94,4568,5154,1399,2794,5155,1463,2025,4569,3218,1944,5156, 828,1105, // 1424
172 4245,1262,1394,5157,4246, 605,4570,5158,1784,2876,5159,2835, 819,2102, 578,2200, // 1440
173 2952,5160,1502, 436,3287,4247,3288,2836,4005,2919,3472,3473,5161,2721,2320,5162, // 1456
174 5163,2337,2068, 23,4571, 193, 826,3838,2103, 699,1630,4248,3098, 390,1794,1064, // 1472
175 3581,5164,1579,3099,3100,1400,5165,4249,1839,1640,2877,5166,4572,4573, 137,4250, // 1488
176 598,3101,1967, 780, 104, 974,2953,5167, 278, 899, 253, 402, 572, 504, 493,1339, // 1504
177 5168,4006,1275,4574,2582,2558,5169,3706,3049,3102,2253, 565,1334,2722, 863, 41, // 1520
178 5170,5171,4575,5172,1657,2338, 19, 463,2760,4251, 606,5173,2999,3289,1087,2085, // 1536
179 1323,2662,3000,5174,1631,1623,1750,4252,2691,5175,2878, 791,2723,2663,2339, 232, // 1552
180 2421,5176,3001,1498,5177,2664,2630, 755,1366,3707,3290,3151,2026,1609, 119,1918, // 1568
181 3474, 862,1026,4253,5178,4007,3839,4576,4008,4577,2265,1952,2477,5179,1125, 817, // 1584
182 4254,4255,4009,1513,1766,2041,1487,4256,3050,3291,2837,3840,3152,5180,5181,1507, // 1600
183 5182,2692, 733, 40,1632,1106,2879, 345,4257, 841,2531, 230,4578,3002,1847,3292, // 1616
184 3475,5183,1263, 986,3476,5184, 735, 879, 254,1137, 857, 622,1300,1180,1388,1562, // 1632
185 4010,4011,2954, 967,2761,2665,1349, 592,2134,1692,3361,3003,1995,4258,1679,4012, // 1648
186 1902,2188,5185, 739,3708,2724,1296,1290,5186,4259,2201,2202,1922,1563,2605,2559, // 1664
187 1871,2762,3004,5187, 435,5188, 343,1108, 596, 17,1751,4579,2239,3477,3709,5189, // 1680
188 4580, 294,3582,2955,1693, 477, 979, 281,2042,3583, 643,2043,3710,2631,2795,2266, // 1696
189 1031,2340,2135,2303,3584,4581, 367,1249,2560,5190,3585,5191,4582,1283,3362,2005, // 1712
190 240,1762,3363,4583,4584, 836,1069,3153, 474,5192,2149,2532, 268,3586,5193,3219, // 1728
191 1521,1284,5194,1658,1546,4260,5195,3587,3588,5196,4261,3364,2693,1685,4262, 961, // 1744
192 1673,2632, 190,2006,2203,3841,4585,4586,5197, 570,2504,3711,1490,5198,4587,2633, // 1760
193 3293,1957,4588, 584,1514, 396,1045,1945,5199,4589,1968,2449,5200,5201,4590,4013, // 1776
194 619,5202,3154,3294, 215,2007,2796,2561,3220,4591,3221,4592, 763,4263,3842,4593, // 1792
195 5203,5204,1958,1767,2956,3365,3712,1174, 452,1477,4594,3366,3155,5205,2838,1253, // 1808
196 2387,2189,1091,2290,4264, 492,5206, 638,1169,1825,2136,1752,4014, 648, 926,1021, // 1824
197 1324,4595, 520,4596, 997, 847,1007, 892,4597,3843,2267,1872,3713,2405,1785,4598, // 1840
198 1953,2957,3103,3222,1728,4265,2044,3714,4599,2008,1701,3156,1551, 30,2268,4266, // 1856
199 5207,2027,4600,3589,5208, 501,5209,4267, 594,3478,2166,1822,3590,3479,3591,3223, // 1872
200 829,2839,4268,5210,1680,3157,1225,4269,5211,3295,4601,4270,3158,2341,5212,4602, // 1888
201 4271,5213,4015,4016,5214,1848,2388,2606,3367,5215,4603, 374,4017, 652,4272,4273, // 1904
202 375,1140, 798,5216,5217,5218,2366,4604,2269, 546,1659, 138,3051,2450,4605,5219, // 1920
203 2254, 612,1849, 910, 796,3844,1740,1371, 825,3845,3846,5220,2920,2562,5221, 692, // 1936
204 444,3052,2634, 801,4606,4274,5222,1491, 244,1053,3053,4275,4276, 340,5223,4018, // 1952
205 1041,3005, 293,1168, 87,1357,5224,1539, 959,5225,2240, 721, 694,4277,3847, 219, // 1968
206 1478, 644,1417,3368,2666,1413,1401,1335,1389,4019,5226,5227,3006,2367,3159,1826, // 1984
207 730,1515, 184,2840, 66,4607,5228,1660,2958, 246,3369, 378,1457, 226,3480, 975, // 2000
208 4020,2959,1264,3592, 674, 696,5229, 163,5230,1141,2422,2167, 713,3593,3370,4608, // 2016
209 4021,5231,5232,1186, 15,5233,1079,1070,5234,1522,3224,3594, 276,1050,2725, 758, // 2032
210 1126, 653,2960,3296,5235,2342, 889,3595,4022,3104,3007, 903,1250,4609,4023,3481, // 2048
211 3596,1342,1681,1718, 766,3297, 286, 89,2961,3715,5236,1713,5237,2607,3371,3008, // 2064
212 5238,2962,2219,3225,2880,5239,4610,2505,2533, 181, 387,1075,4024, 731,2190,3372, // 2080
213 5240,3298, 310, 313,3482,2304, 770,4278, 54,3054, 189,4611,3105,3848,4025,5241, // 2096
214 1230,1617,1850, 355,3597,4279,4612,3373, 111,4280,3716,1350,3160,3483,3055,4281, // 2112
215 2150,3299,3598,5242,2797,4026,4027,3009, 722,2009,5243,1071, 247,1207,2343,2478, // 2128
216 1378,4613,2010, 864,1437,1214,4614, 373,3849,1142,2220, 667,4615, 442,2763,2563, // 2144
217 3850,4028,1969,4282,3300,1840, 837, 170,1107, 934,1336,1883,5244,5245,2119,4283, // 2160
218 2841, 743,1569,5246,4616,4284, 582,2389,1418,3484,5247,1803,5248, 357,1395,1729, // 2176
219 3717,3301,2423,1564,2241,5249,3106,3851,1633,4617,1114,2086,4285,1532,5250, 482, // 2192
220 2451,4618,5251,5252,1492, 833,1466,5253,2726,3599,1641,2842,5254,1526,1272,3718, // 2208
221 4286,1686,1795, 416,2564,1903,1954,1804,5255,3852,2798,3853,1159,2321,5256,2881, // 2224
222 4619,1610,1584,3056,2424,2764, 443,3302,1163,3161,5257,5258,4029,5259,4287,2506, // 2240
223 3057,4620,4030,3162,2104,1647,3600,2011,1873,4288,5260,4289, 431,3485,5261, 250, // 2256
224 97, 81,4290,5262,1648,1851,1558, 160, 848,5263, 866, 740,1694,5264,2204,2843, // 2272
225 3226,4291,4621,3719,1687, 950,2479, 426, 469,3227,3720,3721,4031,5265,5266,1188, // 2288
226 424,1996, 861,3601,4292,3854,2205,2694, 168,1235,3602,4293,5267,2087,1674,4622, // 2304
227 3374,3303, 220,2565,1009,5268,3855, 670,3010, 332,1208, 717,5269,5270,3603,2452, // 2320
228 4032,3375,5271, 513,5272,1209,2882,3376,3163,4623,1080,5273,5274,5275,5276,2534, // 2336
229 3722,3604, 815,1587,4033,4034,5277,3605,3486,3856,1254,4624,1328,3058,1390,4035, // 2352
230 1741,4036,3857,4037,5278, 236,3858,2453,3304,5279,5280,3723,3859,1273,3860,4625, // 2368
231 5281, 308,5282,4626, 245,4627,1852,2480,1307,2583, 430, 715,2137,2454,5283, 270, // 2384
232 199,2883,4038,5284,3606,2727,1753, 761,1754, 725,1661,1841,4628,3487,3724,5285, // 2400
233 5286, 587, 14,3305, 227,2608, 326, 480,2270, 943,2765,3607, 291, 650,1884,5287, // 2416
234 1702,1226, 102,1547, 62,3488, 904,4629,3489,1164,4294,5288,5289,1224,1548,2766, // 2432
235 391, 498,1493,5290,1386,1419,5291,2056,1177,4630, 813, 880,1081,2368, 566,1145, // 2448
236 4631,2291,1001,1035,2566,2609,2242, 394,1286,5292,5293,2069,5294, 86,1494,1730, // 2464
237 4039, 491,1588, 745, 897,2963, 843,3377,4040,2767,2884,3306,1768, 998,2221,2070, // 2480
238 397,1827,1195,1970,3725,3011,3378, 284,5295,3861,2507,2138,2120,1904,5296,4041, // 2496
239 2151,4042,4295,1036,3490,1905, 114,2567,4296, 209,1527,5297,5298,2964,2844,2635, // 2512
240 2390,2728,3164, 812,2568,5299,3307,5300,1559, 737,1885,3726,1210, 885, 28,2695, // 2528
241 3608,3862,5301,4297,1004,1780,4632,5302, 346,1982,2222,2696,4633,3863,1742, 797, // 2544
242 1642,4043,1934,1072,1384,2152, 896,4044,3308,3727,3228,2885,3609,5303,2569,1959, // 2560
243 4634,2455,1786,5304,5305,5306,4045,4298,1005,1308,3728,4299,2729,4635,4636,1528, // 2576
244 2610, 161,1178,4300,1983, 987,4637,1101,4301, 631,4046,1157,3229,2425,1343,1241, // 2592
245 1016,2243,2570, 372, 877,2344,2508,1160, 555,1935, 911,4047,5307, 466,1170, 169, // 2608
246 1051,2921,2697,3729,2481,3012,1182,2012,2571,1251,2636,5308, 992,2345,3491,1540, // 2624
247 2730,1201,2071,2406,1997,2482,5309,4638, 528,1923,2191,1503,1874,1570,2369,3379, // 2640
248 3309,5310, 557,1073,5311,1828,3492,2088,2271,3165,3059,3107, 767,3108,2799,4639, // 2656
249 1006,4302,4640,2346,1267,2179,3730,3230, 778,4048,3231,2731,1597,2667,5312,4641, // 2672
250 5313,3493,5314,5315,5316,3310,2698,1433,3311, 131, 95,1504,4049, 723,4303,3166, // 2688
251 1842,3610,2768,2192,4050,2028,2105,3731,5317,3013,4051,1218,5318,3380,3232,4052, // 2704
252 4304,2584, 248,1634,3864, 912,5319,2845,3732,3060,3865, 654, 53,5320,3014,5321, // 2720
253 1688,4642, 777,3494,1032,4053,1425,5322, 191, 820,2121,2846, 971,4643, 931,3233, // 2736
254 135, 664, 783,3866,1998, 772,2922,1936,4054,3867,4644,2923,3234, 282,2732, 640, // 2752
255 1372,3495,1127, 922, 325,3381,5323,5324, 711,2045,5325,5326,4055,2223,2800,1937, // 2768
256 4056,3382,2224,2255,3868,2305,5327,4645,3869,1258,3312,4057,3235,2139,2965,4058, // 2784
257 4059,5328,2225, 258,3236,4646, 101,1227,5329,3313,1755,5330,1391,3314,5331,2924, // 2800
258 2057, 893,5332,5333,5334,1402,4305,2347,5335,5336,3237,3611,5337,5338, 878,1325, // 2816
259 1781,2801,4647, 259,1385,2585, 744,1183,2272,4648,5339,4060,2509,5340, 684,1024, // 2832
260 4306,5341, 472,3612,3496,1165,3315,4061,4062, 322,2153, 881, 455,1695,1152,1340, // 2848
261 660, 554,2154,4649,1058,4650,4307, 830,1065,3383,4063,4651,1924,5342,1703,1919, // 2864
262 5343, 932,2273, 122,5344,4652, 947, 677,5345,3870,2637, 297,1906,1925,2274,4653, // 2880
263 2322,3316,5346,5347,4308,5348,4309, 84,4310, 112, 989,5349, 547,1059,4064, 701, // 2896
264 3613,1019,5350,4311,5351,3497, 942, 639, 457,2306,2456, 993,2966, 407, 851, 494, // 2912
265 4654,3384, 927,5352,1237,5353,2426,3385, 573,4312, 680, 921,2925,1279,1875, 285, // 2928
266 790,1448,1984, 719,2168,5354,5355,4655,4065,4066,1649,5356,1541, 563,5357,1077, // 2944
267 5358,3386,3061,3498, 511,3015,4067,4068,3733,4069,1268,2572,3387,3238,4656,4657, // 2960
268 5359, 535,1048,1276,1189,2926,2029,3167,1438,1373,2847,2967,1134,2013,5360,4313, // 2976
269 1238,2586,3109,1259,5361, 700,5362,2968,3168,3734,4314,5363,4315,1146,1876,1907, // 2992
270 4658,2611,4070, 781,2427, 132,1589, 203, 147, 273,2802,2407, 898,1787,2155,4071, // 3008
271 4072,5364,3871,2803,5365,5366,4659,4660,5367,3239,5368,1635,3872, 965,5369,1805, // 3024
272 2699,1516,3614,1121,1082,1329,3317,4073,1449,3873, 65,1128,2848,2927,2769,1590, // 3040
273 3874,5370,5371, 12,2668, 45, 976,2587,3169,4661, 517,2535,1013,1037,3240,5372, // 3056
274 3875,2849,5373,3876,5374,3499,5375,2612, 614,1999,2323,3877,3110,2733,2638,5376, // 3072
275 2588,4316, 599,1269,5377,1811,3735,5378,2700,3111, 759,1060, 489,1806,3388,3318, // 3088
276 1358,5379,5380,2391,1387,1215,2639,2256, 490,5381,5382,4317,1759,2392,2348,5383, // 3104
277 4662,3878,1908,4074,2640,1807,3241,4663,3500,3319,2770,2349, 874,5384,5385,3501, // 3120
278 3736,1859, 91,2928,3737,3062,3879,4664,5386,3170,4075,2669,5387,3502,1202,1403, // 3136
279 3880,2969,2536,1517,2510,4665,3503,2511,5388,4666,5389,2701,1886,1495,1731,4076, // 3152
280 2370,4667,5390,2030,5391,5392,4077,2702,1216, 237,2589,4318,2324,4078,3881,4668, // 3168
281 4669,2703,3615,3504, 445,4670,5393,5394,5395,5396,2771, 61,4079,3738,1823,4080, // 3184
282 5397, 687,2046, 935, 925, 405,2670, 703,1096,1860,2734,4671,4081,1877,1367,2704, // 3200
283 3389, 918,2106,1782,2483, 334,3320,1611,1093,4672, 564,3171,3505,3739,3390, 945, // 3216
284 2641,2058,4673,5398,1926, 872,4319,5399,3506,2705,3112, 349,4320,3740,4082,4674, // 3232
285 3882,4321,3741,2156,4083,4675,4676,4322,4677,2408,2047, 782,4084, 400, 251,4323, // 3248
286 1624,5400,5401, 277,3742, 299,1265, 476,1191,3883,2122,4324,4325,1109, 205,5402, // 3264
287 2590,1000,2157,3616,1861,5403,5404,5405,4678,5406,4679,2573, 107,2484,2158,4085, // 3280
288 3507,3172,5407,1533, 541,1301, 158, 753,4326,2886,3617,5408,1696, 370,1088,4327, // 3296
289 4680,3618, 579, 327, 440, 162,2244, 269,1938,1374,3508, 968,3063, 56,1396,3113, // 3312
290 2107,3321,3391,5409,1927,2159,4681,3016,5410,3619,5411,5412,3743,4682,2485,5413, // 3328
291 2804,5414,1650,4683,5415,2613,5416,5417,4086,2671,3392,1149,3393,4087,3884,4088, // 3344
292 5418,1076, 49,5419, 951,3242,3322,3323, 450,2850, 920,5420,1812,2805,2371,4328, // 3360
293 1909,1138,2372,3885,3509,5421,3243,4684,1910,1147,1518,2428,4685,3886,5422,4686, // 3376
294 2393,2614, 260,1796,3244,5423,5424,3887,3324, 708,5425,3620,1704,5426,3621,1351, // 3392
295 1618,3394,3017,1887, 944,4329,3395,4330,3064,3396,4331,5427,3744, 422, 413,1714, // 3408
296 3325, 500,2059,2350,4332,2486,5428,1344,1911, 954,5429,1668,5430,5431,4089,2409, // 3424
297 4333,3622,3888,4334,5432,2307,1318,2512,3114, 133,3115,2887,4687, 629, 31,2851, // 3440
298 2706,3889,4688, 850, 949,4689,4090,2970,1732,2089,4335,1496,1853,5433,4091, 620, // 3456
299 3245, 981,1242,3745,3397,1619,3746,1643,3326,2140,2457,1971,1719,3510,2169,5434, // 3472
300 3246,5435,5436,3398,1829,5437,1277,4690,1565,2048,5438,1636,3623,3116,5439, 869, // 3488
301 2852, 655,3890,3891,3117,4092,3018,3892,1310,3624,4691,5440,5441,5442,1733, 558, // 3504
302 4692,3747, 335,1549,3065,1756,4336,3748,1946,3511,1830,1291,1192, 470,2735,2108, // 3520
303 2806, 913,1054,4093,5443,1027,5444,3066,4094,4693, 982,2672,3399,3173,3512,3247, // 3536
304 3248,1947,2807,5445, 571,4694,5446,1831,5447,3625,2591,1523,2429,5448,2090, 984, // 3552
305 4695,3749,1960,5449,3750, 852, 923,2808,3513,3751, 969,1519, 999,2049,2325,1705, // 3568
306 5450,3118, 615,1662, 151, 597,4095,2410,2326,1049, 275,4696,3752,4337, 568,3753, // 3584
307 3626,2487,4338,3754,5451,2430,2275, 409,3249,5452,1566,2888,3514,1002, 769,2853, // 3600
308 194,2091,3174,3755,2226,3327,4339, 628,1505,5453,5454,1763,2180,3019,4096, 521, // 3616
309 1161,2592,1788,2206,2411,4697,4097,1625,4340,4341, 412, 42,3119, 464,5455,2642, // 3632
310 4698,3400,1760,1571,2889,3515,2537,1219,2207,3893,2643,2141,2373,4699,4700,3328, // 3648
311 1651,3401,3627,5456,5457,3628,2488,3516,5458,3756,5459,5460,2276,2092, 460,5461, // 3664
312 4701,5462,3020, 962, 588,3629, 289,3250,2644,1116, 52,5463,3067,1797,5464,5465, // 3680
313 5466,1467,5467,1598,1143,3757,4342,1985,1734,1067,4702,1280,3402, 465,4703,1572, // 3696
314 510,5468,1928,2245,1813,1644,3630,5469,4704,3758,5470,5471,2673,1573,1534,5472, // 3712
315 5473, 536,1808,1761,3517,3894,3175,2645,5474,5475,5476,4705,3518,2929,1912,2809, // 3728
316 5477,3329,1122, 377,3251,5478, 360,5479,5480,4343,1529, 551,5481,2060,3759,1769, // 3744
317 2431,5482,2930,4344,3330,3120,2327,2109,2031,4706,1404, 136,1468,1479, 672,1171, // 3760
318 3252,2308, 271,3176,5483,2772,5484,2050, 678,2736, 865,1948,4707,5485,2014,4098, // 3776
319 2971,5486,2737,2227,1397,3068,3760,4708,4709,1735,2931,3403,3631,5487,3895, 509, // 3792
320 2854,2458,2890,3896,5488,5489,3177,3178,4710,4345,2538,4711,2309,1166,1010, 552, // 3808
321 681,1888,5490,5491,2972,2973,4099,1287,1596,1862,3179, 358, 453, 736, 175, 478, // 3824
322 1117, 905,1167,1097,5492,1854,1530,5493,1706,5494,2181,3519,2292,3761,3520,3632, // 3840
323 4346,2093,4347,5495,3404,1193,2489,4348,1458,2193,2208,1863,1889,1421,3331,2932, // 3856
324 3069,2182,3521, 595,2123,5496,4100,5497,5498,4349,1707,2646, 223,3762,1359, 751, // 3872
325 3121, 183,3522,5499,2810,3021, 419,2374, 633, 704,3897,2394, 241,5500,5501,5502, // 3888
326 838,3022,3763,2277,2773,2459,3898,1939,2051,4101,1309,3122,2246,1181,5503,1136, // 3904
327 2209,3899,2375,1446,4350,2310,4712,5504,5505,4351,1055,2615, 484,3764,5506,4102, // 3920
328 625,4352,2278,3405,1499,4353,4103,5507,4104,4354,3253,2279,2280,3523,5508,5509, // 3936
329 2774, 808,2616,3765,3406,4105,4355,3123,2539, 526,3407,3900,4356, 955,5510,1620, // 3952
330 4357,2647,2432,5511,1429,3766,1669,1832, 994, 928,5512,3633,1260,5513,5514,5515, // 3968
331 1949,2293, 741,2933,1626,4358,2738,2460, 867,1184, 362,3408,1392,5516,5517,4106, // 3984
332 4359,1770,1736,3254,2934,4713,4714,1929,2707,1459,1158,5518,3070,3409,2891,1292, // 4000
333 1930,2513,2855,3767,1986,1187,2072,2015,2617,4360,5519,2574,2514,2170,3768,2490, // 4016
334 3332,5520,3769,4715,5521,5522, 666,1003,3023,1022,3634,4361,5523,4716,1814,2257, // 4032
335 574,3901,1603, 295,1535, 705,3902,4362, 283, 858, 417,5524,5525,3255,4717,4718, // 4048
336 3071,1220,1890,1046,2281,2461,4107,1393,1599, 689,2575, 388,4363,5526,2491, 802, // 4064
337 5527,2811,3903,2061,1405,2258,5528,4719,3904,2110,1052,1345,3256,1585,5529, 809, // 4080
338 5530,5531,5532, 575,2739,3524, 956,1552,1469,1144,2328,5533,2329,1560,2462,3635, // 4096
339 3257,4108, 616,2210,4364,3180,2183,2294,5534,1833,5535,3525,4720,5536,1319,3770, // 4112
340 3771,1211,3636,1023,3258,1293,2812,5537,5538,5539,3905, 607,2311,3906, 762,2892, // 4128
341 1439,4365,1360,4721,1485,3072,5540,4722,1038,4366,1450,2062,2648,4367,1379,4723, // 4144
342 2593,5541,5542,4368,1352,1414,2330,2935,1172,5543,5544,3907,3908,4724,1798,1451, // 4160
343 5545,5546,5547,5548,2936,4109,4110,2492,2351, 411,4111,4112,3637,3333,3124,4725, // 4176
344 1561,2674,1452,4113,1375,5549,5550, 47,2974, 316,5551,1406,1591,2937,3181,5552, // 4192
345 1025,2142,3125,3182, 354,2740, 884,2228,4369,2412, 508,3772, 726,3638, 996,2433, // 4208
346 3639, 729,5553, 392,2194,1453,4114,4726,3773,5554,5555,2463,3640,2618,1675,2813, // 4224
347 919,2352,2975,2353,1270,4727,4115, 73,5556,5557, 647,5558,3259,2856,2259,1550, // 4240
348 1346,3024,5559,1332, 883,3526,5560,5561,5562,5563,3334,2775,5564,1212, 831,1347, // 4256
349 4370,4728,2331,3909,1864,3073, 720,3910,4729,4730,3911,5565,4371,5566,5567,4731, // 4272
350 5568,5569,1799,4732,3774,2619,4733,3641,1645,2376,4734,5570,2938, 669,2211,2675, // 4288
351 2434,5571,2893,5572,5573,1028,3260,5574,4372,2413,5575,2260,1353,5576,5577,4735, // 4304
352 3183, 518,5578,4116,5579,4373,1961,5580,2143,4374,5581,5582,3025,2354,2355,3912, // 4320
353 516,1834,1454,4117,2708,4375,4736,2229,2620,1972,1129,3642,5583,2776,5584,2976, // 4336
354 1422, 577,1470,3026,1524,3410,5585,5586, 432,4376,3074,3527,5587,2594,1455,2515, // 4352
355 2230,1973,1175,5588,1020,2741,4118,3528,4737,5589,2742,5590,1743,1361,3075,3529, // 4368
356 2649,4119,4377,4738,2295, 895, 924,4378,2171, 331,2247,3076, 166,1627,3077,1098, // 4384
357 5591,1232,2894,2231,3411,4739, 657, 403,1196,2377, 542,3775,3412,1600,4379,3530, // 4400
358 5592,4740,2777,3261, 576, 530,1362,4741,4742,2540,2676,3776,4120,5593, 842,3913, // 4416
359 5594,2814,2032,1014,4121, 213,2709,3413, 665, 621,4380,5595,3777,2939,2435,5596, // 4432
360 2436,3335,3643,3414,4743,4381,2541,4382,4744,3644,1682,4383,3531,1380,5597, 724, // 4448
361 2282, 600,1670,5598,1337,1233,4745,3126,2248,5599,1621,4746,5600, 651,4384,5601, // 4464
362 1612,4385,2621,5602,2857,5603,2743,2312,3078,5604, 716,2464,3079, 174,1255,2710, // 4480
363 4122,3645, 548,1320,1398, 728,4123,1574,5605,1891,1197,3080,4124,5606,3081,3082, // 4496
364 3778,3646,3779, 747,5607, 635,4386,4747,5608,5609,5610,4387,5611,5612,4748,5613, // 4512
365 3415,4749,2437, 451,5614,3780,2542,2073,4388,2744,4389,4125,5615,1764,4750,5616, // 4528
366 4390, 350,4751,2283,2395,2493,5617,4391,4126,2249,1434,4127, 488,4752, 458,4392, // 4544
367 4128,3781, 771,1330,2396,3914,2576,3184,2160,2414,1553,2677,3185,4393,5618,2494, // 4560
368 2895,2622,1720,2711,4394,3416,4753,5619,2543,4395,5620,3262,4396,2778,5621,2016, // 4576
369 2745,5622,1155,1017,3782,3915,5623,3336,2313, 201,1865,4397,1430,5624,4129,5625, // 4592
370 5626,5627,5628,5629,4398,1604,5630, 414,1866, 371,2595,4754,4755,3532,2017,3127, // 4608
371 4756,1708, 960,4399, 887, 389,2172,1536,1663,1721,5631,2232,4130,2356,2940,1580, // 4624
372 5632,5633,1744,4757,2544,4758,4759,5634,4760,5635,2074,5636,4761,3647,3417,2896, // 4640
373 4400,5637,4401,2650,3418,2815, 673,2712,2465, 709,3533,4131,3648,4402,5638,1148, // 4656
374 502, 634,5639,5640,1204,4762,3649,1575,4763,2623,3783,5641,3784,3128, 948,3263, // 4672
375 121,1745,3916,1110,5642,4403,3083,2516,3027,4132,3785,1151,1771,3917,1488,4133, // 4688
376 1987,5643,2438,3534,5644,5645,2094,5646,4404,3918,1213,1407,2816, 531,2746,2545, // 4704
377 3264,1011,1537,4764,2779,4405,3129,1061,5647,3786,3787,1867,2897,5648,2018, 120, // 4720
378 4406,4407,2063,3650,3265,2314,3919,2678,3419,1955,4765,4134,5649,3535,1047,2713, // 4736
379 1266,5650,1368,4766,2858, 649,3420,3920,2546,2747,1102,2859,2679,5651,5652,2000, // 4752
380 5653,1111,3651,2977,5654,2495,3921,3652,2817,1855,3421,3788,5655,5656,3422,2415, // 4768
381 2898,3337,3266,3653,5657,2577,5658,3654,2818,4135,1460, 856,5659,3655,5660,2899, // 4784
382 2978,5661,2900,3922,5662,4408, 632,2517, 875,3923,1697,3924,2296,5663,5664,4767, // 4800
383 3028,1239, 580,4768,4409,5665, 914, 936,2075,1190,4136,1039,2124,5666,5667,5668, // 4816
384 5669,3423,1473,5670,1354,4410,3925,4769,2173,3084,4137, 915,3338,4411,4412,3339, // 4832
385 1605,1835,5671,2748, 398,3656,4413,3926,4138, 328,1913,2860,4139,3927,1331,4414, // 4848
386 3029, 937,4415,5672,3657,4140,4141,3424,2161,4770,3425, 524, 742, 538,3085,1012, // 4864
387 5673,5674,3928,2466,5675, 658,1103, 225,3929,5676,5677,4771,5678,4772,5679,3267, // 4880
388 1243,5680,4142, 963,2250,4773,5681,2714,3658,3186,5682,5683,2596,2332,5684,4774, // 4896
389 5685,5686,5687,3536, 957,3426,2547,2033,1931,2941,2467, 870,2019,3659,1746,2780, // 4912
390 2781,2439,2468,5688,3930,5689,3789,3130,3790,3537,3427,3791,5690,1179,3086,5691, // 4928
391 3187,2378,4416,3792,2548,3188,3131,2749,4143,5692,3428,1556,2549,2297, 977,2901, // 4944
392 2034,4144,1205,3429,5693,1765,3430,3189,2125,1271, 714,1689,4775,3538,5694,2333, // 4960
393 3931, 533,4417,3660,2184, 617,5695,2469,3340,3539,2315,5696,5697,3190,5698,5699, // 4976
394 3932,1988, 618, 427,2651,3540,3431,5700,5701,1244,1690,5702,2819,4418,4776,5703, // 4992
395 3541,4777,5704,2284,1576, 473,3661,4419,3432, 972,5705,3662,5706,3087,5707,5708, // 5008
396 4778,4779,5709,3793,4145,4146,5710, 153,4780, 356,5711,1892,2902,4420,2144, 408, // 5024
397 803,2357,5712,3933,5713,4421,1646,2578,2518,4781,4782,3934,5714,3935,4422,5715, // 5040
398 2416,3433, 752,5716,5717,1962,3341,2979,5718, 746,3030,2470,4783,4423,3794, 698, // 5056
399 4784,1893,4424,3663,2550,4785,3664,3936,5719,3191,3434,5720,1824,1302,4147,2715, // 5072
400 3937,1974,4425,5721,4426,3192, 823,1303,1288,1236,2861,3542,4148,3435, 774,3938, // 5088
401 5722,1581,4786,1304,2862,3939,4787,5723,2440,2162,1083,3268,4427,4149,4428, 344, // 5104
402 1173, 288,2316, 454,1683,5724,5725,1461,4788,4150,2597,5726,5727,4789, 985, 894, // 5120
403 5728,3436,3193,5729,1914,2942,3795,1989,5730,2111,1975,5731,4151,5732,2579,1194, // 5136
404 425,5733,4790,3194,1245,3796,4429,5734,5735,2863,5736, 636,4791,1856,3940, 760, // 5152
405 1800,5737,4430,2212,1508,4792,4152,1894,1684,2298,5738,5739,4793,4431,4432,2213, // 5168
406 479,5740,5741, 832,5742,4153,2496,5743,2980,2497,3797, 990,3132, 627,1815,2652, // 5184
407 4433,1582,4434,2126,2112,3543,4794,5744, 799,4435,3195,5745,4795,2113,1737,3031, // 5200
408 1018, 543, 754,4436,3342,1676,4796,4797,4154,4798,1489,5746,3544,5747,2624,2903, // 5216
409 4155,5748,5749,2981,5750,5751,5752,5753,3196,4799,4800,2185,1722,5754,3269,3270, // 5232
410 1843,3665,1715, 481, 365,1976,1857,5755,5756,1963,2498,4801,5757,2127,3666,3271, // 5248
411 433,1895,2064,2076,5758, 602,2750,5759,5760,5761,5762,5763,3032,1628,3437,5764, // 5264
412 3197,4802,4156,2904,4803,2519,5765,2551,2782,5766,5767,5768,3343,4804,2905,5769, // 5280
413 4805,5770,2864,4806,4807,1221,2982,4157,2520,5771,5772,5773,1868,1990,5774,5775, // 5296
414 5776,1896,5777,5778,4808,1897,4158, 318,5779,2095,4159,4437,5780,5781, 485,5782, // 5312
415 938,3941, 553,2680, 116,5783,3942,3667,5784,3545,2681,2783,3438,3344,2820,5785, // 5328
416 3668,2943,4160,1747,2944,2983,5786,5787, 207,5788,4809,5789,4810,2521,5790,3033, // 5344
417 890,3669,3943,5791,1878,3798,3439,5792,2186,2358,3440,1652,5793,5794,5795, 941, // 5360
418 2299, 208,3546,4161,2020, 330,4438,3944,2906,2499,3799,4439,4811,5796,5797,5798, // 5376 //last 512
419
420 /***************************************************************************************
421 *Everything below is of no interest for detection purpose *
422 ***************************************************************************************
423
424 2522,1613,4812,5799,3345,3945,2523,5800,4162,5801,1637,4163,2471,4813,3946,5802, // 5392
425 2500,3034,3800,5803,5804,2195,4814,5805,2163,5806,5807,5808,5809,5810,5811,5812, // 5408
426 5813,5814,5815,5816,5817,5818,5819,5820,5821,5822,5823,5824,5825,5826,5827,5828, // 5424
427 5829,5830,5831,5832,5833,5834,5835,5836,5837,5838,5839,5840,5841,5842,5843,5844, // 5440
428 5845,5846,5847,5848,5849,5850,5851,5852,5853,5854,5855,5856,5857,5858,5859,5860, // 5456
429 5861,5862,5863,5864,5865,5866,5867,5868,5869,5870,5871,5872,5873,5874,5875,5876, // 5472
430 5877,5878,5879,5880,5881,5882,5883,5884,5885,5886,5887,5888,5889,5890,5891,5892, // 5488
431 5893,5894,5895,5896,5897,5898,5899,5900,5901,5902,5903,5904,5905,5906,5907,5908, // 5504
432 5909,5910,5911,5912,5913,5914,5915,5916,5917,5918,5919,5920,5921,5922,5923,5924, // 5520
433 5925,5926,5927,5928,5929,5930,5931,5932,5933,5934,5935,5936,5937,5938,5939,5940, // 5536
434 5941,5942,5943,5944,5945,5946,5947,5948,5949,5950,5951,5952,5953,5954,5955,5956, // 5552
435 5957,5958,5959,5960,5961,5962,5963,5964,5965,5966,5967,5968,5969,5970,5971,5972, // 5568
436 5973,5974,5975,5976,5977,5978,5979,5980,5981,5982,5983,5984,5985,5986,5987,5988, // 5584
437 5989,5990,5991,5992,5993,5994,5995,5996,5997,5998,5999,6000,6001,6002,6003,6004, // 5600
438 6005,6006,6007,6008,6009,6010,6011,6012,6013,6014,6015,6016,6017,6018,6019,6020, // 5616
439 6021,6022,6023,6024,6025,6026,6027,6028,6029,6030,6031,6032,6033,6034,6035,6036, // 5632
440 6037,6038,6039,6040,6041,6042,6043,6044,6045,6046,6047,6048,6049,6050,6051,6052, // 5648
441 6053,6054,6055,6056,6057,6058,6059,6060,6061,6062,6063,6064,6065,6066,6067,6068, // 5664
442 6069,6070,6071,6072,6073,6074,6075,6076,6077,6078,6079,6080,6081,6082,6083,6084, // 5680
443 6085,6086,6087,6088,6089,6090,6091,6092,6093,6094,6095,6096,6097,6098,6099,6100, // 5696
444 6101,6102,6103,6104,6105,6106,6107,6108,6109,6110,6111,6112,6113,6114,6115,6116, // 5712
445 6117,6118,6119,6120,6121,6122,6123,6124,6125,6126,6127,6128,6129,6130,6131,6132, // 5728
446 6133,6134,6135,6136,6137,6138,6139,6140,6141,6142,6143,6144,6145,6146,6147,6148, // 5744
447 6149,6150,6151,6152,6153,6154,6155,6156,6157,6158,6159,6160,6161,6162,6163,6164, // 5760
448 6165,6166,6167,6168,6169,6170,6171,6172,6173,6174,6175,6176,6177,6178,6179,6180, // 5776
449 6181,6182,6183,6184,6185,6186,6187,6188,6189,6190,6191,6192,6193,6194,6195,6196, // 5792
450 6197,6198,6199,6200,6201,6202,6203,6204,6205,6206,6207,6208,6209,6210,6211,6212, // 5808
451 6213,6214,6215,6216,6217,6218,6219,6220,6221,6222,6223,3670,6224,6225,6226,6227, // 5824
452 6228,6229,6230,6231,6232,6233,6234,6235,6236,6237,6238,6239,6240,6241,6242,6243, // 5840
453 6244,6245,6246,6247,6248,6249,6250,6251,6252,6253,6254,6255,6256,6257,6258,6259, // 5856
454 6260,6261,6262,6263,6264,6265,6266,6267,6268,6269,6270,6271,6272,6273,6274,6275, // 5872
455 6276,6277,6278,6279,6280,6281,6282,6283,6284,6285,4815,6286,6287,6288,6289,6290, // 5888
456 6291,6292,4816,6293,6294,6295,6296,6297,6298,6299,6300,6301,6302,6303,6304,6305, // 5904
457 6306,6307,6308,6309,6310,6311,4817,4818,6312,6313,6314,6315,6316,6317,6318,4819, // 5920
458 6319,6320,6321,6322,6323,6324,6325,6326,6327,6328,6329,6330,6331,6332,6333,6334, // 5936
459 6335,6336,6337,4820,6338,6339,6340,6341,6342,6343,6344,6345,6346,6347,6348,6349, // 5952
460 6350,6351,6352,6353,6354,6355,6356,6357,6358,6359,6360,6361,6362,6363,6364,6365, // 5968
461 6366,6367,6368,6369,6370,6371,6372,6373,6374,6375,6376,6377,6378,6379,6380,6381, // 5984
462 6382,6383,6384,6385,6386,6387,6388,6389,6390,6391,6392,6393,6394,6395,6396,6397, // 6000
463 6398,6399,6400,6401,6402,6403,6404,6405,6406,6407,6408,6409,6410,3441,6411,6412, // 6016
464 6413,6414,6415,6416,6417,6418,6419,6420,6421,6422,6423,6424,6425,4440,6426,6427, // 6032
465 6428,6429,6430,6431,6432,6433,6434,6435,6436,6437,6438,6439,6440,6441,6442,6443, // 6048
466 6444,6445,6446,6447,6448,6449,6450,6451,6452,6453,6454,4821,6455,6456,6457,6458, // 6064
467 6459,6460,6461,6462,6463,6464,6465,6466,6467,6468,6469,6470,6471,6472,6473,6474, // 6080
468 6475,6476,6477,3947,3948,6478,6479,6480,6481,3272,4441,6482,6483,6484,6485,4442, // 6096
469 6486,6487,6488,6489,6490,6491,6492,6493,6494,6495,6496,4822,6497,6498,6499,6500, // 6112
470 6501,6502,6503,6504,6505,6506,6507,6508,6509,6510,6511,6512,6513,6514,6515,6516, // 6128
471 6517,6518,6519,6520,6521,6522,6523,6524,6525,6526,6527,6528,6529,6530,6531,6532, // 6144
472 6533,6534,6535,6536,6537,6538,6539,6540,6541,6542,6543,6544,6545,6546,6547,6548, // 6160
473 6549,6550,6551,6552,6553,6554,6555,6556,2784,6557,4823,6558,6559,6560,6561,6562, // 6176
474 6563,6564,6565,6566,6567,6568,6569,3949,6570,6571,6572,4824,6573,6574,6575,6576, // 6192
475 6577,6578,6579,6580,6581,6582,6583,4825,6584,6585,6586,3950,2785,6587,6588,6589, // 6208
476 6590,6591,6592,6593,6594,6595,6596,6597,6598,6599,6600,6601,6602,6603,6604,6605, // 6224
477 6606,6607,6608,6609,6610,6611,6612,4826,6613,6614,6615,4827,6616,6617,6618,6619, // 6240
478 6620,6621,6622,6623,6624,6625,4164,6626,6627,6628,6629,6630,6631,6632,6633,6634, // 6256
479 3547,6635,4828,6636,6637,6638,6639,6640,6641,6642,3951,2984,6643,6644,6645,6646, // 6272
480 6647,6648,6649,4165,6650,4829,6651,6652,4830,6653,6654,6655,6656,6657,6658,6659, // 6288
481 6660,6661,6662,4831,6663,6664,6665,6666,6667,6668,6669,6670,6671,4166,6672,4832, // 6304
482 3952,6673,6674,6675,6676,4833,6677,6678,6679,4167,6680,6681,6682,3198,6683,6684, // 6320
483 6685,6686,6687,6688,6689,6690,6691,6692,6693,6694,6695,6696,6697,4834,6698,6699, // 6336
484 6700,6701,6702,6703,6704,6705,6706,6707,6708,6709,6710,6711,6712,6713,6714,6715, // 6352
485 6716,6717,6718,6719,6720,6721,6722,6723,6724,6725,6726,6727,6728,6729,6730,6731, // 6368
486 6732,6733,6734,4443,6735,6736,6737,6738,6739,6740,6741,6742,6743,6744,6745,4444, // 6384
487 6746,6747,6748,6749,6750,6751,6752,6753,6754,6755,6756,6757,6758,6759,6760,6761, // 6400
488 6762,6763,6764,6765,6766,6767,6768,6769,6770,6771,6772,6773,6774,6775,6776,6777, // 6416
489 6778,6779,6780,6781,4168,6782,6783,3442,6784,6785,6786,6787,6788,6789,6790,6791, // 6432
490 4169,6792,6793,6794,6795,6796,6797,6798,6799,6800,6801,6802,6803,6804,6805,6806, // 6448
491 6807,6808,6809,6810,6811,4835,6812,6813,6814,4445,6815,6816,4446,6817,6818,6819, // 6464
492 6820,6821,6822,6823,6824,6825,6826,6827,6828,6829,6830,6831,6832,6833,6834,6835, // 6480
493 3548,6836,6837,6838,6839,6840,6841,6842,6843,6844,6845,6846,4836,6847,6848,6849, // 6496
494 6850,6851,6852,6853,6854,3953,6855,6856,6857,6858,6859,6860,6861,6862,6863,6864, // 6512
495 6865,6866,6867,6868,6869,6870,6871,6872,6873,6874,6875,6876,6877,3199,6878,6879, // 6528
496 6880,6881,6882,4447,6883,6884,6885,6886,6887,6888,6889,6890,6891,6892,6893,6894, // 6544
497 6895,6896,6897,6898,6899,6900,6901,6902,6903,6904,4170,6905,6906,6907,6908,6909, // 6560
498 6910,6911,6912,6913,6914,6915,6916,6917,6918,6919,6920,6921,6922,6923,6924,6925, // 6576
499 6926,6927,4837,6928,6929,6930,6931,6932,6933,6934,6935,6936,3346,6937,6938,4838, // 6592
500 6939,6940,6941,4448,6942,6943,6944,6945,6946,4449,6947,6948,6949,6950,6951,6952, // 6608
501 6953,6954,6955,6956,6957,6958,6959,6960,6961,6962,6963,6964,6965,6966,6967,6968, // 6624
502 6969,6970,6971,6972,6973,6974,6975,6976,6977,6978,6979,6980,6981,6982,6983,6984, // 6640
503 6985,6986,6987,6988,6989,6990,6991,6992,6993,6994,3671,6995,6996,6997,6998,4839, // 6656
504 6999,7000,7001,7002,3549,7003,7004,7005,7006,7007,7008,7009,7010,7011,7012,7013, // 6672
505 7014,7015,7016,7017,7018,7019,7020,7021,7022,7023,7024,7025,7026,7027,7028,7029, // 6688
506 7030,4840,7031,7032,7033,7034,7035,7036,7037,7038,4841,7039,7040,7041,7042,7043, // 6704
507 7044,7045,7046,7047,7048,7049,7050,7051,7052,7053,7054,7055,7056,7057,7058,7059, // 6720
508 7060,7061,7062,7063,7064,7065,7066,7067,7068,7069,7070,2985,7071,7072,7073,7074, // 6736
509 7075,7076,7077,7078,7079,7080,4842,7081,7082,7083,7084,7085,7086,7087,7088,7089, // 6752
510 7090,7091,7092,7093,7094,7095,7096,7097,7098,7099,7100,7101,7102,7103,7104,7105, // 6768
511 7106,7107,7108,7109,7110,7111,7112,7113,7114,7115,7116,7117,7118,4450,7119,7120, // 6784
512 7121,7122,7123,7124,7125,7126,7127,7128,7129,7130,7131,7132,7133,7134,7135,7136, // 6800
513 7137,7138,7139,7140,7141,7142,7143,4843,7144,7145,7146,7147,7148,7149,7150,7151, // 6816
514 7152,7153,7154,7155,7156,7157,7158,7159,7160,7161,7162,7163,7164,7165,7166,7167, // 6832
515 7168,7169,7170,7171,7172,7173,7174,7175,7176,7177,7178,7179,7180,7181,7182,7183, // 6848
516 7184,7185,7186,7187,7188,4171,4172,7189,7190,7191,7192,7193,7194,7195,7196,7197, // 6864
517 7198,7199,7200,7201,7202,7203,7204,7205,7206,7207,7208,7209,7210,7211,7212,7213, // 6880
518 7214,7215,7216,7217,7218,7219,7220,7221,7222,7223,7224,7225,7226,7227,7228,7229, // 6896
519 7230,7231,7232,7233,7234,7235,7236,7237,7238,7239,7240,7241,7242,7243,7244,7245, // 6912
520 7246,7247,7248,7249,7250,7251,7252,7253,7254,7255,7256,7257,7258,7259,7260,7261, // 6928
521 7262,7263,7264,7265,7266,7267,7268,7269,7270,7271,7272,7273,7274,7275,7276,7277, // 6944
522 7278,7279,7280,7281,7282,7283,7284,7285,7286,7287,7288,7289,7290,7291,7292,7293, // 6960
523 7294,7295,7296,4844,7297,7298,7299,7300,7301,7302,7303,7304,7305,7306,7307,7308, // 6976
524 7309,7310,7311,7312,7313,7314,7315,7316,4451,7317,7318,7319,7320,7321,7322,7323, // 6992
525 7324,7325,7326,7327,7328,7329,7330,7331,7332,7333,7334,7335,7336,7337,7338,7339, // 7008
526 7340,7341,7342,7343,7344,7345,7346,7347,7348,7349,7350,7351,7352,7353,4173,7354, // 7024
527 7355,4845,7356,7357,7358,7359,7360,7361,7362,7363,7364,7365,7366,7367,7368,7369, // 7040
528 7370,7371,7372,7373,7374,7375,7376,7377,7378,7379,7380,7381,7382,7383,7384,7385, // 7056
529 7386,7387,7388,4846,7389,7390,7391,7392,7393,7394,7395,7396,7397,7398,7399,7400, // 7072
530 7401,7402,7403,7404,7405,3672,7406,7407,7408,7409,7410,7411,7412,7413,7414,7415, // 7088
531 7416,7417,7418,7419,7420,7421,7422,7423,7424,7425,7426,7427,7428,7429,7430,7431, // 7104
532 7432,7433,7434,7435,7436,7437,7438,7439,7440,7441,7442,7443,7444,7445,7446,7447, // 7120
533 7448,7449,7450,7451,7452,7453,4452,7454,3200,7455,7456,7457,7458,7459,7460,7461, // 7136
534 7462,7463,7464,7465,7466,7467,7468,7469,7470,7471,7472,7473,7474,4847,7475,7476, // 7152
535 7477,3133,7478,7479,7480,7481,7482,7483,7484,7485,7486,7487,7488,7489,7490,7491, // 7168
536 7492,7493,7494,7495,7496,7497,7498,7499,7500,7501,7502,3347,7503,7504,7505,7506, // 7184
537 7507,7508,7509,7510,7511,7512,7513,7514,7515,7516,7517,7518,7519,7520,7521,4848, // 7200
538 7522,7523,7524,7525,7526,7527,7528,7529,7530,7531,7532,7533,7534,7535,7536,7537, // 7216
539 7538,7539,7540,7541,7542,7543,7544,7545,7546,7547,7548,7549,3801,4849,7550,7551, // 7232
540 7552,7553,7554,7555,7556,7557,7558,7559,7560,7561,7562,7563,7564,7565,7566,7567, // 7248
541 7568,7569,3035,7570,7571,7572,7573,7574,7575,7576,7577,7578,7579,7580,7581,7582, // 7264
542 7583,7584,7585,7586,7587,7588,7589,7590,7591,7592,7593,7594,7595,7596,7597,7598, // 7280
543 7599,7600,7601,7602,7603,7604,7605,7606,7607,7608,7609,7610,7611,7612,7613,7614, // 7296
544 7615,7616,4850,7617,7618,3802,7619,7620,7621,7622,7623,7624,7625,7626,7627,7628, // 7312
545 7629,7630,7631,7632,4851,7633,7634,7635,7636,7637,7638,7639,7640,7641,7642,7643, // 7328
546 7644,7645,7646,7647,7648,7649,7650,7651,7652,7653,7654,7655,7656,7657,7658,7659, // 7344
547 7660,7661,7662,7663,7664,7665,7666,7667,7668,7669,7670,4453,7671,7672,7673,7674, // 7360
548 7675,7676,7677,7678,7679,7680,7681,7682,7683,7684,7685,7686,7687,7688,7689,7690, // 7376
549 7691,7692,7693,7694,7695,7696,7697,3443,7698,7699,7700,7701,7702,4454,7703,7704, // 7392
550 7705,7706,7707,7708,7709,7710,7711,7712,7713,2472,7714,7715,7716,7717,7718,7719, // 7408
551 7720,7721,7722,7723,7724,7725,7726,7727,7728,7729,7730,7731,3954,7732,7733,7734, // 7424
552 7735,7736,7737,7738,7739,7740,7741,7742,7743,7744,7745,7746,7747,7748,7749,7750, // 7440
553 3134,7751,7752,4852,7753,7754,7755,4853,7756,7757,7758,7759,7760,4174,7761,7762, // 7456
554 7763,7764,7765,7766,7767,7768,7769,7770,7771,7772,7773,7774,7775,7776,7777,7778, // 7472
555 7779,7780,7781,7782,7783,7784,7785,7786,7787,7788,7789,7790,7791,7792,7793,7794, // 7488
556 7795,7796,7797,7798,7799,7800,7801,7802,7803,7804,7805,4854,7806,7807,7808,7809, // 7504
557 7810,7811,7812,7813,7814,7815,7816,7817,7818,7819,7820,7821,7822,7823,7824,7825, // 7520
558 4855,7826,7827,7828,7829,7830,7831,7832,7833,7834,7835,7836,7837,7838,7839,7840, // 7536
559 7841,7842,7843,7844,7845,7846,7847,3955,7848,7849,7850,7851,7852,7853,7854,7855, // 7552
560 7856,7857,7858,7859,7860,3444,7861,7862,7863,7864,7865,7866,7867,7868,7869,7870, // 7568
561 7871,7872,7873,7874,7875,7876,7877,7878,7879,7880,7881,7882,7883,7884,7885,7886, // 7584
562 7887,7888,7889,7890,7891,4175,7892,7893,7894,7895,7896,4856,4857,7897,7898,7899, // 7600
563 7900,2598,7901,7902,7903,7904,7905,7906,7907,7908,4455,7909,7910,7911,7912,7913, // 7616
564 7914,3201,7915,7916,7917,7918,7919,7920,7921,4858,7922,7923,7924,7925,7926,7927, // 7632
565 7928,7929,7930,7931,7932,7933,7934,7935,7936,7937,7938,7939,7940,7941,7942,7943, // 7648
566 7944,7945,7946,7947,7948,7949,7950,7951,7952,7953,7954,7955,7956,7957,7958,7959, // 7664
567 7960,7961,7962,7963,7964,7965,7966,7967,7968,7969,7970,7971,7972,7973,7974,7975, // 7680
568 7976,7977,7978,7979,7980,7981,4859,7982,7983,7984,7985,7986,7987,7988,7989,7990, // 7696
569 7991,7992,7993,7994,7995,7996,4860,7997,7998,7999,8000,8001,8002,8003,8004,8005, // 7712
570 8006,8007,8008,8009,8010,8011,8012,8013,8014,8015,8016,4176,8017,8018,8019,8020, // 7728
571 8021,8022,8023,4861,8024,8025,8026,8027,8028,8029,8030,8031,8032,8033,8034,8035, // 7744
572 8036,4862,4456,8037,8038,8039,8040,4863,8041,8042,8043,8044,8045,8046,8047,8048, // 7760
573 8049,8050,8051,8052,8053,8054,8055,8056,8057,8058,8059,8060,8061,8062,8063,8064, // 7776
574 8065,8066,8067,8068,8069,8070,8071,8072,8073,8074,8075,8076,8077,8078,8079,8080, // 7792
575 8081,8082,8083,8084,8085,8086,8087,8088,8089,8090,8091,8092,8093,8094,8095,8096, // 7808
576 8097,8098,8099,4864,4177,8100,8101,8102,8103,8104,8105,8106,8107,8108,8109,8110, // 7824
577 8111,8112,8113,8114,8115,8116,8117,8118,8119,8120,4178,8121,8122,8123,8124,8125, // 7840
578 8126,8127,8128,8129,8130,8131,8132,8133,8134,8135,8136,8137,8138,8139,8140,8141, // 7856
579 8142,8143,8144,8145,4865,4866,8146,8147,8148,8149,8150,8151,8152,8153,8154,8155, // 7872
580 8156,8157,8158,8159,8160,8161,8162,8163,8164,8165,4179,8166,8167,8168,8169,8170, // 7888
581 8171,8172,8173,8174,8175,8176,8177,8178,8179,8180,8181,4457,8182,8183,8184,8185, // 7904
582 8186,8187,8188,8189,8190,8191,8192,8193,8194,8195,8196,8197,8198,8199,8200,8201, // 7920
583 8202,8203,8204,8205,8206,8207,8208,8209,8210,8211,8212,8213,8214,8215,8216,8217, // 7936
584 8218,8219,8220,8221,8222,8223,8224,8225,8226,8227,8228,8229,8230,8231,8232,8233, // 7952
585 8234,8235,8236,8237,8238,8239,8240,8241,8242,8243,8244,8245,8246,8247,8248,8249, // 7968
586 8250,8251,8252,8253,8254,8255,8256,3445,8257,8258,8259,8260,8261,8262,4458,8263, // 7984
587 8264,8265,8266,8267,8268,8269,8270,8271,8272,4459,8273,8274,8275,8276,3550,8277, // 8000
588 8278,8279,8280,8281,8282,8283,8284,8285,8286,8287,8288,8289,4460,8290,8291,8292, // 8016
589 8293,8294,8295,8296,8297,8298,8299,8300,8301,8302,8303,8304,8305,8306,8307,4867, // 8032
590 8308,8309,8310,8311,8312,3551,8313,8314,8315,8316,8317,8318,8319,8320,8321,8322, // 8048
591 8323,8324,8325,8326,4868,8327,8328,8329,8330,8331,8332,8333,8334,8335,8336,8337, // 8064
592 8338,8339,8340,8341,8342,8343,8344,8345,8346,8347,8348,8349,8350,8351,8352,8353, // 8080
593 8354,8355,8356,8357,8358,8359,8360,8361,8362,8363,4869,4461,8364,8365,8366,8367, // 8096
594 8368,8369,8370,4870,8371,8372,8373,8374,8375,8376,8377,8378,8379,8380,8381,8382, // 8112
595 8383,8384,8385,8386,8387,8388,8389,8390,8391,8392,8393,8394,8395,8396,8397,8398, // 8128
596 8399,8400,8401,8402,8403,8404,8405,8406,8407,8408,8409,8410,4871,8411,8412,8413, // 8144
597 8414,8415,8416,8417,8418,8419,8420,8421,8422,4462,8423,8424,8425,8426,8427,8428, // 8160
598 8429,8430,8431,8432,8433,2986,8434,8435,8436,8437,8438,8439,8440,8441,8442,8443, // 8176
599 8444,8445,8446,8447,8448,8449,8450,8451,8452,8453,8454,8455,8456,8457,8458,8459, // 8192
600 8460,8461,8462,8463,8464,8465,8466,8467,8468,8469,8470,8471,8472,8473,8474,8475, // 8208
601 8476,8477,8478,4180,8479,8480,8481,8482,8483,8484,8485,8486,8487,8488,8489,8490, // 8224
602 8491,8492,8493,8494,8495,8496,8497,8498,8499,8500,8501,8502,8503,8504,8505,8506, // 8240
603 8507,8508,8509,8510,8511,8512,8513,8514,8515,8516,8517,8518,8519,8520,8521,8522, // 8256
604 8523,8524,8525,8526,8527,8528,8529,8530,8531,8532,8533,8534,8535,8536,8537,8538, // 8272
605 8539,8540,8541,8542,8543,8544,8545,8546,8547,8548,8549,8550,8551,8552,8553,8554, // 8288
606 8555,8556,8557,8558,8559,8560,8561,8562,8563,8564,4872,8565,8566,8567,8568,8569, // 8304
607 8570,8571,8572,8573,4873,8574,8575,8576,8577,8578,8579,8580,8581,8582,8583,8584, // 8320
608 8585,8586,8587,8588,8589,8590,8591,8592,8593,8594,8595,8596,8597,8598,8599,8600, // 8336
609 8601,8602,8603,8604,8605,3803,8606,8607,8608,8609,8610,8611,8612,8613,4874,3804, // 8352
610 8614,8615,8616,8617,8618,8619,8620,8621,3956,8622,8623,8624,8625,8626,8627,8628, // 8368
611 8629,8630,8631,8632,8633,8634,8635,8636,8637,8638,2865,8639,8640,8641,8642,8643, // 8384
612 8644,8645,8646,8647,8648,8649,8650,8651,8652,8653,8654,8655,8656,4463,8657,8658, // 8400
613 8659,4875,4876,8660,8661,8662,8663,8664,8665,8666,8667,8668,8669,8670,8671,8672, // 8416
614 8673,8674,8675,8676,8677,8678,8679,8680,8681,4464,8682,8683,8684,8685,8686,8687, // 8432
615 8688,8689,8690,8691,8692,8693,8694,8695,8696,8697,8698,8699,8700,8701,8702,8703, // 8448
616 8704,8705,8706,8707,8708,8709,2261,8710,8711,8712,8713,8714,8715,8716,8717,8718, // 8464
617 8719,8720,8721,8722,8723,8724,8725,8726,8727,8728,8729,8730,8731,8732,8733,4181, // 8480
618 8734,8735,8736,8737,8738,8739,8740,8741,8742,8743,8744,8745,8746,8747,8748,8749, // 8496
619 8750,8751,8752,8753,8754,8755,8756,8757,8758,8759,8760,8761,8762,8763,4877,8764, // 8512
620 8765,8766,8767,8768,8769,8770,8771,8772,8773,8774,8775,8776,8777,8778,8779,8780, // 8528
621 8781,8782,8783,8784,8785,8786,8787,8788,4878,8789,4879,8790,8791,8792,4880,8793, // 8544
622 8794,8795,8796,8797,8798,8799,8800,8801,4881,8802,8803,8804,8805,8806,8807,8808, // 8560
623 8809,8810,8811,8812,8813,8814,8815,3957,8816,8817,8818,8819,8820,8821,8822,8823, // 8576
624 8824,8825,8826,8827,8828,8829,8830,8831,8832,8833,8834,8835,8836,8837,8838,8839, // 8592
625 8840,8841,8842,8843,8844,8845,8846,8847,4882,8848,8849,8850,8851,8852,8853,8854, // 8608
626 8855,8856,8857,8858,8859,8860,8861,8862,8863,8864,8865,8866,8867,8868,8869,8870, // 8624
627 8871,8872,8873,8874,8875,8876,8877,8878,8879,8880,8881,8882,8883,8884,3202,8885, // 8640
628 8886,8887,8888,8889,8890,8891,8892,8893,8894,8895,8896,8897,8898,8899,8900,8901, // 8656
629 8902,8903,8904,8905,8906,8907,8908,8909,8910,8911,8912,8913,8914,8915,8916,8917, // 8672
630 8918,8919,8920,8921,8922,8923,8924,4465,8925,8926,8927,8928,8929,8930,8931,8932, // 8688
631 4883,8933,8934,8935,8936,8937,8938,8939,8940,8941,8942,8943,2214,8944,8945,8946, // 8704
632 8947,8948,8949,8950,8951,8952,8953,8954,8955,8956,8957,8958,8959,8960,8961,8962, // 8720
633 8963,8964,8965,4884,8966,8967,8968,8969,8970,8971,8972,8973,8974,8975,8976,8977, // 8736
634 8978,8979,8980,8981,8982,8983,8984,8985,8986,8987,8988,8989,8990,8991,8992,4885, // 8752
635 8993,8994,8995,8996,8997,8998,8999,9000,9001,9002,9003,9004,9005,9006,9007,9008, // 8768
636 9009,9010,9011,9012,9013,9014,9015,9016,9017,9018,9019,9020,9021,4182,9022,9023, // 8784
637 9024,9025,9026,9027,9028,9029,9030,9031,9032,9033,9034,9035,9036,9037,9038,9039, // 8800
638 9040,9041,9042,9043,9044,9045,9046,9047,9048,9049,9050,9051,9052,9053,9054,9055, // 8816
639 9056,9057,9058,9059,9060,9061,9062,9063,4886,9064,9065,9066,9067,9068,9069,4887, // 8832
640 9070,9071,9072,9073,9074,9075,9076,9077,9078,9079,9080,9081,9082,9083,9084,9085, // 8848
641 9086,9087,9088,9089,9090,9091,9092,9093,9094,9095,9096,9097,9098,9099,9100,9101, // 8864
642 9102,9103,9104,9105,9106,9107,9108,9109,9110,9111,9112,9113,9114,9115,9116,9117, // 8880
643 9118,9119,9120,9121,9122,9123,9124,9125,9126,9127,9128,9129,9130,9131,9132,9133, // 8896
644 9134,9135,9136,9137,9138,9139,9140,9141,3958,9142,9143,9144,9145,9146,9147,9148, // 8912
645 9149,9150,9151,4888,9152,9153,9154,9155,9156,9157,9158,9159,9160,9161,9162,9163, // 8928
646 9164,9165,9166,9167,9168,9169,9170,9171,9172,9173,9174,9175,4889,9176,9177,9178, // 8944
647 9179,9180,9181,9182,9183,9184,9185,9186,9187,9188,9189,9190,9191,9192,9193,9194, // 8960
648 9195,9196,9197,9198,9199,9200,9201,9202,9203,4890,9204,9205,9206,9207,9208,9209, // 8976
649 9210,9211,9212,9213,9214,9215,9216,9217,9218,9219,9220,9221,9222,4466,9223,9224, // 8992
650 9225,9226,9227,9228,9229,9230,9231,9232,9233,9234,9235,9236,9237,9238,9239,9240, // 9008
651 9241,9242,9243,9244,9245,4891,9246,9247,9248,9249,9250,9251,9252,9253,9254,9255, // 9024
652 9256,9257,4892,9258,9259,9260,9261,4893,4894,9262,9263,9264,9265,9266,9267,9268, // 9040
653 9269,9270,9271,9272,9273,4467,9274,9275,9276,9277,9278,9279,9280,9281,9282,9283, // 9056
654 9284,9285,3673,9286,9287,9288,9289,9290,9291,9292,9293,9294,9295,9296,9297,9298, // 9072
655 9299,9300,9301,9302,9303,9304,9305,9306,9307,9308,9309,9310,9311,9312,9313,9314, // 9088
656 9315,9316,9317,9318,9319,9320,9321,9322,4895,9323,9324,9325,9326,9327,9328,9329, // 9104
657 9330,9331,9332,9333,9334,9335,9336,9337,9338,9339,9340,9341,9342,9343,9344,9345, // 9120
658 9346,9347,4468,9348,9349,9350,9351,9352,9353,9354,9355,9356,9357,9358,9359,9360, // 9136
659 9361,9362,9363,9364,9365,9366,9367,9368,9369,9370,9371,9372,9373,4896,9374,4469, // 9152
660 9375,9376,9377,9378,9379,4897,9380,9381,9382,9383,9384,9385,9386,9387,9388,9389, // 9168
661 9390,9391,9392,9393,9394,9395,9396,9397,9398,9399,9400,9401,9402,9403,9404,9405, // 9184
662 9406,4470,9407,2751,9408,9409,3674,3552,9410,9411,9412,9413,9414,9415,9416,9417, // 9200
663 9418,9419,9420,9421,4898,9422,9423,9424,9425,9426,9427,9428,9429,3959,9430,9431, // 9216
664 9432,9433,9434,9435,9436,4471,9437,9438,9439,9440,9441,9442,9443,9444,9445,9446, // 9232
665 9447,9448,9449,9450,3348,9451,9452,9453,9454,9455,9456,9457,9458,9459,9460,9461, // 9248
666 9462,9463,9464,9465,9466,9467,9468,9469,9470,9471,9472,4899,9473,9474,9475,9476, // 9264
667 9477,4900,9478,9479,9480,9481,9482,9483,9484,9485,9486,9487,9488,3349,9489,9490, // 9280
668 9491,9492,9493,9494,9495,9496,9497,9498,9499,9500,9501,9502,9503,9504,9505,9506, // 9296
669 9507,9508,9509,9510,9511,9512,9513,9514,9515,9516,9517,9518,9519,9520,4901,9521, // 9312
670 9522,9523,9524,9525,9526,4902,9527,9528,9529,9530,9531,9532,9533,9534,9535,9536, // 9328
671 9537,9538,9539,9540,9541,9542,9543,9544,9545,9546,9547,9548,9549,9550,9551,9552, // 9344
672 9553,9554,9555,9556,9557,9558,9559,9560,9561,9562,9563,9564,9565,9566,9567,9568, // 9360
673 9569,9570,9571,9572,9573,9574,9575,9576,9577,9578,9579,9580,9581,9582,9583,9584, // 9376
674 3805,9585,9586,9587,9588,9589,9590,9591,9592,9593,9594,9595,9596,9597,9598,9599, // 9392
675 9600,9601,9602,4903,9603,9604,9605,9606,9607,4904,9608,9609,9610,9611,9612,9613, // 9408
676 9614,4905,9615,9616,9617,9618,9619,9620,9621,9622,9623,9624,9625,9626,9627,9628, // 9424
677 9629,9630,9631,9632,4906,9633,9634,9635,9636,9637,9638,9639,9640,9641,9642,9643, // 9440
678 4907,9644,9645,9646,9647,9648,9649,9650,9651,9652,9653,9654,9655,9656,9657,9658, // 9456
679 9659,9660,9661,9662,9663,9664,9665,9666,9667,9668,9669,9670,9671,9672,4183,9673, // 9472
680 9674,9675,9676,9677,4908,9678,9679,9680,9681,4909,9682,9683,9684,9685,9686,9687, // 9488
681 9688,9689,9690,4910,9691,9692,9693,3675,9694,9695,9696,2945,9697,9698,9699,9700, // 9504
682 9701,9702,9703,9704,9705,4911,9706,9707,9708,9709,9710,9711,9712,9713,9714,9715, // 9520
683 9716,9717,9718,9719,9720,9721,9722,9723,9724,9725,9726,9727,9728,9729,9730,9731, // 9536
684 9732,9733,9734,9735,4912,9736,9737,9738,9739,9740,4913,9741,9742,9743,9744,9745, // 9552
685 9746,9747,9748,9749,9750,9751,9752,9753,9754,9755,9756,9757,9758,4914,9759,9760, // 9568
686 9761,9762,9763,9764,9765,9766,9767,9768,9769,9770,9771,9772,9773,9774,9775,9776, // 9584
687 9777,9778,9779,9780,9781,9782,4915,9783,9784,9785,9786,9787,9788,9789,9790,9791, // 9600
688 9792,9793,4916,9794,9795,9796,9797,9798,9799,9800,9801,9802,9803,9804,9805,9806, // 9616
689 9807,9808,9809,9810,9811,9812,9813,9814,9815,9816,9817,9818,9819,9820,9821,9822, // 9632
690 9823,9824,9825,9826,9827,9828,9829,9830,9831,9832,9833,9834,9835,9836,9837,9838, // 9648
691 9839,9840,9841,9842,9843,9844,9845,9846,9847,9848,9849,9850,9851,9852,9853,9854, // 9664
692 9855,9856,9857,9858,9859,9860,9861,9862,9863,9864,9865,9866,9867,9868,4917,9869, // 9680
693 9870,9871,9872,9873,9874,9875,9876,9877,9878,9879,9880,9881,9882,9883,9884,9885, // 9696
694 9886,9887,9888,9889,9890,9891,9892,4472,9893,9894,9895,9896,9897,3806,9898,9899, // 9712
695 9900,9901,9902,9903,9904,9905,9906,9907,9908,9909,9910,9911,9912,9913,9914,4918, // 9728
696 9915,9916,9917,4919,9918,9919,9920,9921,4184,9922,9923,9924,9925,9926,9927,9928, // 9744
697 9929,9930,9931,9932,9933,9934,9935,9936,9937,9938,9939,9940,9941,9942,9943,9944, // 9760
698 9945,9946,4920,9947,9948,9949,9950,9951,9952,9953,9954,9955,4185,9956,9957,9958, // 9776
699 9959,9960,9961,9962,9963,9964,9965,4921,9966,9967,9968,4473,9969,9970,9971,9972, // 9792
700 9973,9974,9975,9976,9977,4474,9978,9979,9980,9981,9982,9983,9984,9985,9986,9987, // 9808
701 9988,9989,9990,9991,9992,9993,9994,9995,9996,9997,9998,9999,10000,10001,10002,10003, // 9824
702 10004,10005,10006,10007,10008,10009,10010,10011,10012,10013,10014,10015,10016,10017,10018,10019, // 9840
703 10020,10021,4922,10022,4923,10023,10024,10025,10026,10027,10028,10029,10030,10031,10032,10033, // 9856
704 10034,10035,10036,10037,10038,10039,10040,10041,10042,10043,10044,10045,10046,10047,10048,4924, // 9872
705 10049,10050,10051,10052,10053,10054,10055,10056,10057,10058,10059,10060,10061,10062,10063,10064, // 9888
706 10065,10066,10067,10068,10069,10070,10071,10072,10073,10074,10075,10076,10077,10078,10079,10080, // 9904
707 10081,10082,10083,10084,10085,10086,10087,4475,10088,10089,10090,10091,10092,10093,10094,10095, // 9920
708 10096,10097,4476,10098,10099,10100,10101,10102,10103,10104,10105,10106,10107,10108,10109,10110, // 9936
709 10111,2174,10112,10113,10114,10115,10116,10117,10118,10119,10120,10121,10122,10123,10124,10125, // 9952
710 10126,10127,10128,10129,10130,10131,10132,10133,10134,10135,10136,10137,10138,10139,10140,3807, // 9968
711 4186,4925,10141,10142,10143,10144,10145,10146,10147,4477,4187,10148,10149,10150,10151,10152, // 9984
712 10153,4188,10154,10155,10156,10157,10158,10159,10160,10161,4926,10162,10163,10164,10165,10166, //10000
713 10167,10168,10169,10170,10171,10172,10173,10174,10175,10176,10177,10178,10179,10180,10181,10182, //10016
714 10183,10184,10185,10186,10187,10188,10189,10190,10191,10192,3203,10193,10194,10195,10196,10197, //10032
715 10198,10199,10200,4478,10201,10202,10203,10204,4479,10205,10206,10207,10208,10209,10210,10211, //10048
716 10212,10213,10214,10215,10216,10217,10218,10219,10220,10221,10222,10223,10224,10225,10226,10227, //10064
717 10228,10229,10230,10231,10232,10233,10234,4927,10235,10236,10237,10238,10239,10240,10241,10242, //10080
718 10243,10244,10245,10246,10247,10248,10249,10250,10251,10252,10253,10254,10255,10256,10257,10258, //10096
719 10259,10260,10261,10262,10263,10264,10265,10266,10267,10268,10269,10270,10271,10272,10273,4480, //10112
720 4928,4929,10274,10275,10276,10277,10278,10279,10280,10281,10282,10283,10284,10285,10286,10287, //10128
721 10288,10289,10290,10291,10292,10293,10294,10295,10296,10297,10298,10299,10300,10301,10302,10303, //10144
722 10304,10305,10306,10307,10308,10309,10310,10311,10312,10313,10314,10315,10316,10317,10318,10319, //10160
723 10320,10321,10322,10323,10324,10325,10326,10327,10328,10329,10330,10331,10332,10333,10334,4930, //10176
724 10335,10336,10337,10338,10339,10340,10341,10342,4931,10343,10344,10345,10346,10347,10348,10349, //10192
725 10350,10351,10352,10353,10354,10355,3088,10356,2786,10357,10358,10359,10360,4189,10361,10362, //10208
726 10363,10364,10365,10366,10367,10368,10369,10370,10371,10372,10373,10374,10375,4932,10376,10377, //10224
727 10378,10379,10380,10381,10382,10383,10384,10385,10386,10387,10388,10389,10390,10391,10392,4933, //10240
728 10393,10394,10395,4934,10396,10397,10398,10399,10400,10401,10402,10403,10404,10405,10406,10407, //10256
729 10408,10409,10410,10411,10412,3446,10413,10414,10415,10416,10417,10418,10419,10420,10421,10422, //10272
730 10423,4935,10424,10425,10426,10427,10428,10429,10430,4936,10431,10432,10433,10434,10435,10436, //10288
731 10437,10438,10439,10440,10441,10442,10443,4937,10444,10445,10446,10447,4481,10448,10449,10450, //10304
732 10451,10452,10453,10454,10455,10456,10457,10458,10459,10460,10461,10462,10463,10464,10465,10466, //10320
733 10467,10468,10469,10470,10471,10472,10473,10474,10475,10476,10477,10478,10479,10480,10481,10482, //10336
734 10483,10484,10485,10486,10487,10488,10489,10490,10491,10492,10493,10494,10495,10496,10497,10498, //10352
735 10499,10500,10501,10502,10503,10504,10505,4938,10506,10507,10508,10509,10510,2552,10511,10512, //10368
736 10513,10514,10515,10516,3447,10517,10518,10519,10520,10521,10522,10523,10524,10525,10526,10527, //10384
737 10528,10529,10530,10531,10532,10533,10534,10535,10536,10537,10538,10539,10540,10541,10542,10543, //10400
738 4482,10544,4939,10545,10546,10547,10548,10549,10550,10551,10552,10553,10554,10555,10556,10557, //10416
739 10558,10559,10560,10561,10562,10563,10564,10565,10566,10567,3676,4483,10568,10569,10570,10571, //10432
740 10572,3448,10573,10574,10575,10576,10577,10578,10579,10580,10581,10582,10583,10584,10585,10586, //10448
741 10587,10588,10589,10590,10591,10592,10593,10594,10595,10596,10597,10598,10599,10600,10601,10602, //10464
742 10603,10604,10605,10606,10607,10608,10609,10610,10611,10612,10613,10614,10615,10616,10617,10618, //10480
743 10619,10620,10621,10622,10623,10624,10625,10626,10627,4484,10628,10629,10630,10631,10632,4940, //10496
744 10633,10634,10635,10636,10637,10638,10639,10640,10641,10642,10643,10644,10645,10646,10647,10648, //10512
745 10649,10650,10651,10652,10653,10654,10655,10656,4941,10657,10658,10659,2599,10660,10661,10662, //10528
746 10663,10664,10665,10666,3089,10667,10668,10669,10670,10671,10672,10673,10674,10675,10676,10677, //10544
747 10678,10679,10680,4942,10681,10682,10683,10684,10685,10686,10687,10688,10689,10690,10691,10692, //10560
748 10693,10694,10695,10696,10697,4485,10698,10699,10700,10701,10702,10703,10704,4943,10705,3677, //10576
749 10706,10707,10708,10709,10710,10711,10712,4944,10713,10714,10715,10716,10717,10718,10719,10720, //10592
750 10721,10722,10723,10724,10725,10726,10727,10728,4945,10729,10730,10731,10732,10733,10734,10735, //10608
751 10736,10737,10738,10739,10740,10741,10742,10743,10744,10745,10746,10747,10748,10749,10750,10751, //10624
752 10752,10753,10754,10755,10756,10757,10758,10759,10760,10761,4946,10762,10763,10764,10765,10766, //10640
753 10767,4947,4948,10768,10769,10770,10771,10772,10773,10774,10775,10776,10777,10778,10779,10780, //10656
754 10781,10782,10783,10784,10785,10786,10787,10788,10789,10790,10791,10792,10793,10794,10795,10796, //10672
755 10797,10798,10799,10800,10801,10802,10803,10804,10805,10806,10807,10808,10809,10810,10811,10812, //10688
756 10813,10814,10815,10816,10817,10818,10819,10820,10821,10822,10823,10824,10825,10826,10827,10828, //10704
757 10829,10830,10831,10832,10833,10834,10835,10836,10837,10838,10839,10840,10841,10842,10843,10844, //10720
758 10845,10846,10847,10848,10849,10850,10851,10852,10853,10854,10855,10856,10857,10858,10859,10860, //10736
759 10861,10862,10863,10864,10865,10866,10867,10868,10869,10870,10871,10872,10873,10874,10875,10876, //10752
760 10877,10878,4486,10879,10880,10881,10882,10883,10884,10885,4949,10886,10887,10888,10889,10890, //10768
761 10891,10892,10893,10894,10895,10896,10897,10898,10899,10900,10901,10902,10903,10904,10905,10906, //10784
762 10907,10908,10909,10910,10911,10912,10913,10914,10915,10916,10917,10918,10919,4487,10920,10921, //10800
763 10922,10923,10924,10925,10926,10927,10928,10929,10930,10931,10932,4950,10933,10934,10935,10936, //10816
764 10937,10938,10939,10940,10941,10942,10943,10944,10945,10946,10947,10948,10949,4488,10950,10951, //10832
765 10952,10953,10954,10955,10956,10957,10958,10959,4190,10960,10961,10962,10963,10964,10965,10966, //10848
766 10967,10968,10969,10970,10971,10972,10973,10974,10975,10976,10977,10978,10979,10980,10981,10982, //10864
767 10983,10984,10985,10986,10987,10988,10989,10990,10991,10992,10993,10994,10995,10996,10997,10998, //10880
768 10999,11000,11001,11002,11003,11004,11005,11006,3960,11007,11008,11009,11010,11011,11012,11013, //10896
769 11014,11015,11016,11017,11018,11019,11020,11021,11022,11023,11024,11025,11026,11027,11028,11029, //10912
770 11030,11031,11032,4951,11033,11034,11035,11036,11037,11038,11039,11040,11041,11042,11043,11044, //10928
771 11045,11046,11047,4489,11048,11049,11050,11051,4952,11052,11053,11054,11055,11056,11057,11058, //10944
772 4953,11059,11060,11061,11062,11063,11064,11065,11066,11067,11068,11069,11070,11071,4954,11072, //10960
773 11073,11074,11075,11076,11077,11078,11079,11080,11081,11082,11083,11084,11085,11086,11087,11088, //10976
774 11089,11090,11091,11092,11093,11094,11095,11096,11097,11098,11099,11100,11101,11102,11103,11104, //10992
775 11105,11106,11107,11108,11109,11110,11111,11112,11113,11114,11115,3808,11116,11117,11118,11119, //11008
776 11120,11121,11122,11123,11124,11125,11126,11127,11128,11129,11130,11131,11132,11133,11134,4955, //11024
777 11135,11136,11137,11138,11139,11140,11141,11142,11143,11144,11145,11146,11147,11148,11149,11150, //11040
778 11151,11152,11153,11154,11155,11156,11157,11158,11159,11160,11161,4956,11162,11163,11164,11165, //11056
779 11166,11167,11168,11169,11170,11171,11172,11173,11174,11175,11176,11177,11178,11179,11180,4957, //11072
780 11181,11182,11183,11184,11185,11186,4958,11187,11188,11189,11190,11191,11192,11193,11194,11195, //11088
781 11196,11197,11198,11199,11200,3678,11201,11202,11203,11204,11205,11206,4191,11207,11208,11209, //11104
782 11210,11211,11212,11213,11214,11215,11216,11217,11218,11219,11220,11221,11222,11223,11224,11225, //11120
783 11226,11227,11228,11229,11230,11231,11232,11233,11234,11235,11236,11237,11238,11239,11240,11241, //11136
784 11242,11243,11244,11245,11246,11247,11248,11249,11250,11251,4959,11252,11253,11254,11255,11256, //11152
785 11257,11258,11259,11260,11261,11262,11263,11264,11265,11266,11267,11268,11269,11270,11271,11272, //11168
786 11273,11274,11275,11276,11277,11278,11279,11280,11281,11282,11283,11284,11285,11286,11287,11288, //11184
787 11289,11290,11291,11292,11293,11294,11295,11296,11297,11298,11299,11300,11301,11302,11303,11304, //11200
788 11305,11306,11307,11308,11309,11310,11311,11312,11313,11314,3679,11315,11316,11317,11318,4490, //11216
789 11319,11320,11321,11322,11323,11324,11325,11326,11327,11328,11329,11330,11331,11332,11333,11334, //11232
790 11335,11336,11337,11338,11339,11340,11341,11342,11343,11344,11345,11346,11347,4960,11348,11349, //11248
791 11350,11351,11352,11353,11354,11355,11356,11357,11358,11359,11360,11361,11362,11363,11364,11365, //11264
792 11366,11367,11368,11369,11370,11371,11372,11373,11374,11375,11376,11377,3961,4961,11378,11379, //11280
793 11380,11381,11382,11383,11384,11385,11386,11387,11388,11389,11390,11391,11392,11393,11394,11395, //11296
794 11396,11397,4192,11398,11399,11400,11401,11402,11403,11404,11405,11406,11407,11408,11409,11410, //11312
795 11411,4962,11412,11413,11414,11415,11416,11417,11418,11419,11420,11421,11422,11423,11424,11425, //11328
796 11426,11427,11428,11429,11430,11431,11432,11433,11434,11435,11436,11437,11438,11439,11440,11441, //11344
797 11442,11443,11444,11445,11446,11447,11448,11449,11450,11451,11452,11453,11454,11455,11456,11457, //11360
798 11458,11459,11460,11461,11462,11463,11464,11465,11466,11467,11468,11469,4963,11470,11471,4491, //11376
799 11472,11473,11474,11475,4964,11476,11477,11478,11479,11480,11481,11482,11483,11484,11485,11486, //11392
800 11487,11488,11489,11490,11491,11492,4965,11493,11494,11495,11496,11497,11498,11499,11500,11501, //11408
801 11502,11503,11504,11505,11506,11507,11508,11509,11510,11511,11512,11513,11514,11515,11516,11517, //11424
802 11518,11519,11520,11521,11522,11523,11524,11525,11526,11527,11528,11529,3962,11530,11531,11532, //11440
803 11533,11534,11535,11536,11537,11538,11539,11540,11541,11542,11543,11544,11545,11546,11547,11548, //11456
804 11549,11550,11551,11552,11553,11554,11555,11556,11557,11558,11559,11560,11561,11562,11563,11564, //11472
805 4193,4194,11565,11566,11567,11568,11569,11570,11571,11572,11573,11574,11575,11576,11577,11578, //11488
806 11579,11580,11581,11582,11583,11584,11585,11586,11587,11588,11589,11590,11591,4966,4195,11592, //11504
807 11593,11594,11595,11596,11597,11598,11599,11600,11601,11602,11603,11604,3090,11605,11606,11607, //11520
808 11608,11609,11610,4967,11611,11612,11613,11614,11615,11616,11617,11618,11619,11620,11621,11622, //11536
809 11623,11624,11625,11626,11627,11628,11629,11630,11631,11632,11633,11634,11635,11636,11637,11638, //11552
810 11639,11640,11641,11642,11643,11644,11645,11646,11647,11648,11649,11650,11651,11652,11653,11654, //11568
811 11655,11656,11657,11658,11659,11660,11661,11662,11663,11664,11665,11666,11667,11668,11669,11670, //11584
812 11671,11672,11673,11674,4968,11675,11676,11677,11678,11679,11680,11681,11682,11683,11684,11685, //11600
813 11686,11687,11688,11689,11690,11691,11692,11693,3809,11694,11695,11696,11697,11698,11699,11700, //11616
814 11701,11702,11703,11704,11705,11706,11707,11708,11709,11710,11711,11712,11713,11714,11715,11716, //11632
815 11717,11718,3553,11719,11720,11721,11722,11723,11724,11725,11726,11727,11728,11729,11730,4969, //11648
816 11731,11732,11733,11734,11735,11736,11737,11738,11739,11740,4492,11741,11742,11743,11744,11745, //11664
817 11746,11747,11748,11749,11750,11751,11752,4970,11753,11754,11755,11756,11757,11758,11759,11760, //11680
818 11761,11762,11763,11764,11765,11766,11767,11768,11769,11770,11771,11772,11773,11774,11775,11776, //11696
819 11777,11778,11779,11780,11781,11782,11783,11784,11785,11786,11787,11788,11789,11790,4971,11791, //11712
820 11792,11793,11794,11795,11796,11797,4972,11798,11799,11800,11801,11802,11803,11804,11805,11806, //11728
821 11807,11808,11809,11810,4973,11811,11812,11813,11814,11815,11816,11817,11818,11819,11820,11821, //11744
822 11822,11823,11824,11825,11826,11827,11828,11829,11830,11831,11832,11833,11834,3680,3810,11835, //11760
823 11836,4974,11837,11838,11839,11840,11841,11842,11843,11844,11845,11846,11847,11848,11849,11850, //11776
824 11851,11852,11853,11854,11855,11856,11857,11858,11859,11860,11861,11862,11863,11864,11865,11866, //11792
825 11867,11868,11869,11870,11871,11872,11873,11874,11875,11876,11877,11878,11879,11880,11881,11882, //11808
826 11883,11884,4493,11885,11886,11887,11888,11889,11890,11891,11892,11893,11894,11895,11896,11897, //11824
827 11898,11899,11900,11901,11902,11903,11904,11905,11906,11907,11908,11909,11910,11911,11912,11913, //11840
828 11914,11915,4975,11916,11917,11918,11919,11920,11921,11922,11923,11924,11925,11926,11927,11928, //11856
829 11929,11930,11931,11932,11933,11934,11935,11936,11937,11938,11939,11940,11941,11942,11943,11944, //11872
830 11945,11946,11947,11948,11949,4976,11950,11951,11952,11953,11954,11955,11956,11957,11958,11959, //11888
831 11960,11961,11962,11963,11964,11965,11966,11967,11968,11969,11970,11971,11972,11973,11974,11975, //11904
832 11976,11977,11978,11979,11980,11981,11982,11983,11984,11985,11986,11987,4196,11988,11989,11990, //11920
833 11991,11992,4977,11993,11994,11995,11996,11997,11998,11999,12000,12001,12002,12003,12004,12005, //11936
834 12006,12007,12008,12009,12010,12011,12012,12013,12014,12015,12016,12017,12018,12019,12020,12021, //11952
835 12022,12023,12024,12025,12026,12027,12028,12029,12030,12031,12032,12033,12034,12035,12036,12037, //11968
836 12038,12039,12040,12041,12042,12043,12044,12045,12046,12047,12048,12049,12050,12051,12052,12053, //11984
837 12054,12055,12056,12057,12058,12059,12060,12061,4978,12062,12063,12064,12065,12066,12067,12068, //12000
838 12069,12070,12071,12072,12073,12074,12075,12076,12077,12078,12079,12080,12081,12082,12083,12084, //12016
839 12085,12086,12087,12088,12089,12090,12091,12092,12093,12094,12095,12096,12097,12098,12099,12100, //12032
840 12101,12102,12103,12104,12105,12106,12107,12108,12109,12110,12111,12112,12113,12114,12115,12116, //12048
841 12117,12118,12119,12120,12121,12122,12123,4979,12124,12125,12126,12127,12128,4197,12129,12130, //12064
842 12131,12132,12133,12134,12135,12136,12137,12138,12139,12140,12141,12142,12143,12144,12145,12146, //12080
843 12147,12148,12149,12150,12151,12152,12153,12154,4980,12155,12156,12157,12158,12159,12160,4494, //12096
844 12161,12162,12163,12164,3811,12165,12166,12167,12168,12169,4495,12170,12171,4496,12172,12173, //12112
845 12174,12175,12176,3812,12177,12178,12179,12180,12181,12182,12183,12184,12185,12186,12187,12188, //12128
846 12189,12190,12191,12192,12193,12194,12195,12196,12197,12198,12199,12200,12201,12202,12203,12204, //12144
847 12205,12206,12207,12208,12209,12210,12211,12212,12213,12214,12215,12216,12217,12218,12219,12220, //12160
848 12221,4981,12222,12223,12224,12225,12226,12227,12228,12229,12230,12231,12232,12233,12234,12235, //12176
849 4982,12236,12237,12238,12239,12240,12241,12242,12243,12244,12245,4983,12246,12247,12248,12249, //12192
850 4984,12250,12251,12252,12253,12254,12255,12256,12257,12258,12259,12260,12261,12262,12263,12264, //12208
851 4985,12265,4497,12266,12267,12268,12269,12270,12271,12272,12273,12274,12275,12276,12277,12278, //12224
852 12279,12280,12281,12282,12283,12284,12285,12286,12287,4986,12288,12289,12290,12291,12292,12293, //12240
853 12294,12295,12296,2473,12297,12298,12299,12300,12301,12302,12303,12304,12305,12306,12307,12308, //12256
854 12309,12310,12311,12312,12313,12314,12315,12316,12317,12318,12319,3963,12320,12321,12322,12323, //12272
855 12324,12325,12326,12327,12328,12329,12330,12331,12332,4987,12333,12334,12335,12336,12337,12338, //12288
856 12339,12340,12341,12342,12343,12344,12345,12346,12347,12348,12349,12350,12351,12352,12353,12354, //12304
857 12355,12356,12357,12358,12359,3964,12360,12361,12362,12363,12364,12365,12366,12367,12368,12369, //12320
858 12370,3965,12371,12372,12373,12374,12375,12376,12377,12378,12379,12380,12381,12382,12383,12384, //12336
859 12385,12386,12387,12388,12389,12390,12391,12392,12393,12394,12395,12396,12397,12398,12399,12400, //12352
860 12401,12402,12403,12404,12405,12406,12407,12408,4988,12409,12410,12411,12412,12413,12414,12415, //12368
861 12416,12417,12418,12419,12420,12421,12422,12423,12424,12425,12426,12427,12428,12429,12430,12431, //12384
862 12432,12433,12434,12435,12436,12437,12438,3554,12439,12440,12441,12442,12443,12444,12445,12446, //12400
863 12447,12448,12449,12450,12451,12452,12453,12454,12455,12456,12457,12458,12459,12460,12461,12462, //12416
864 12463,12464,4989,12465,12466,12467,12468,12469,12470,12471,12472,12473,12474,12475,12476,12477, //12432
865 12478,12479,12480,4990,12481,12482,12483,12484,12485,12486,12487,12488,12489,4498,12490,12491, //12448
866 12492,12493,12494,12495,12496,12497,12498,12499,12500,12501,12502,12503,12504,12505,12506,12507, //12464
867 12508,12509,12510,12511,12512,12513,12514,12515,12516,12517,12518,12519,12520,12521,12522,12523, //12480
868 12524,12525,12526,12527,12528,12529,12530,12531,12532,12533,12534,12535,12536,12537,12538,12539, //12496
869 12540,12541,12542,12543,12544,12545,12546,12547,12548,12549,12550,12551,4991,12552,12553,12554, //12512
870 12555,12556,12557,12558,12559,12560,12561,12562,12563,12564,12565,12566,12567,12568,12569,12570, //12528
871 12571,12572,12573,12574,12575,12576,12577,12578,3036,12579,12580,12581,12582,12583,3966,12584, //12544
872 12585,12586,12587,12588,12589,12590,12591,12592,12593,12594,12595,12596,12597,12598,12599,12600, //12560
873 12601,12602,12603,12604,12605,12606,12607,12608,12609,12610,12611,12612,12613,12614,12615,12616, //12576
874 12617,12618,12619,12620,12621,12622,12623,12624,12625,12626,12627,12628,12629,12630,12631,12632, //12592
875 12633,12634,12635,12636,12637,12638,12639,12640,12641,12642,12643,12644,12645,12646,4499,12647, //12608
876 12648,12649,12650,12651,12652,12653,12654,12655,12656,12657,12658,12659,12660,12661,12662,12663, //12624
877 12664,12665,12666,12667,12668,12669,12670,12671,12672,12673,12674,12675,12676,12677,12678,12679, //12640
878 12680,12681,12682,12683,12684,12685,12686,12687,12688,12689,12690,12691,12692,12693,12694,12695, //12656
879 12696,12697,12698,4992,12699,12700,12701,12702,12703,12704,12705,12706,12707,12708,12709,12710, //12672
880 12711,12712,12713,12714,12715,12716,12717,12718,12719,12720,12721,12722,12723,12724,12725,12726, //12688
881 12727,12728,12729,12730,12731,12732,12733,12734,12735,12736,12737,12738,12739,12740,12741,12742, //12704
882 12743,12744,12745,12746,12747,12748,12749,12750,12751,12752,12753,12754,12755,12756,12757,12758, //12720
883 12759,12760,12761,12762,12763,12764,12765,12766,12767,12768,12769,12770,12771,12772,12773,12774, //12736
884 12775,12776,12777,12778,4993,2175,12779,12780,12781,12782,12783,12784,12785,12786,4500,12787, //12752
885 12788,12789,12790,12791,12792,12793,12794,12795,12796,12797,12798,12799,12800,12801,12802,12803, //12768
886 12804,12805,12806,12807,12808,12809,12810,12811,12812,12813,12814,12815,12816,12817,12818,12819, //12784
887 12820,12821,12822,12823,12824,12825,12826,4198,3967,12827,12828,12829,12830,12831,12832,12833, //12800
888 12834,12835,12836,12837,12838,12839,12840,12841,12842,12843,12844,12845,12846,12847,12848,12849, //12816
889 12850,12851,12852,12853,12854,12855,12856,12857,12858,12859,12860,12861,4199,12862,12863,12864, //12832
890 12865,12866,12867,12868,12869,12870,12871,12872,12873,12874,12875,12876,12877,12878,12879,12880, //12848
891 12881,12882,12883,12884,12885,12886,12887,4501,12888,12889,12890,12891,12892,12893,12894,12895, //12864
892 12896,12897,12898,12899,12900,12901,12902,12903,12904,12905,12906,12907,12908,12909,12910,12911, //12880
893 12912,4994,12913,12914,12915,12916,12917,12918,12919,12920,12921,12922,12923,12924,12925,12926, //12896
894 12927,12928,12929,12930,12931,12932,12933,12934,12935,12936,12937,12938,12939,12940,12941,12942, //12912
895 12943,12944,12945,12946,12947,12948,12949,12950,12951,12952,12953,12954,12955,12956,1772,12957, //12928
896 12958,12959,12960,12961,12962,12963,12964,12965,12966,12967,12968,12969,12970,12971,12972,12973, //12944
897 12974,12975,12976,12977,12978,12979,12980,12981,12982,12983,12984,12985,12986,12987,12988,12989, //12960
898 12990,12991,12992,12993,12994,12995,12996,12997,4502,12998,4503,12999,13000,13001,13002,13003, //12976
899 4504,13004,13005,13006,13007,13008,13009,13010,13011,13012,13013,13014,13015,13016,13017,13018, //12992
900 13019,13020,13021,13022,13023,13024,13025,13026,13027,13028,13029,3449,13030,13031,13032,13033, //13008
901 13034,13035,13036,13037,13038,13039,13040,13041,13042,13043,13044,13045,13046,13047,13048,13049, //13024
902 13050,13051,13052,13053,13054,13055,13056,13057,13058,13059,13060,13061,13062,13063,13064,13065, //13040
903 13066,13067,13068,13069,13070,13071,13072,13073,13074,13075,13076,13077,13078,13079,13080,13081, //13056
904 13082,13083,13084,13085,13086,13087,13088,13089,13090,13091,13092,13093,13094,13095,13096,13097, //13072
905 13098,13099,13100,13101,13102,13103,13104,13105,13106,13107,13108,13109,13110,13111,13112,13113, //13088
906 13114,13115,13116,13117,13118,3968,13119,4995,13120,13121,13122,13123,13124,13125,13126,13127, //13104
907 4505,13128,13129,13130,13131,13132,13133,13134,4996,4506,13135,13136,13137,13138,13139,4997, //13120
908 13140,13141,13142,13143,13144,13145,13146,13147,13148,13149,13150,13151,13152,13153,13154,13155, //13136
909 13156,13157,13158,13159,4998,13160,13161,13162,13163,13164,13165,13166,13167,13168,13169,13170, //13152
910 13171,13172,13173,13174,13175,13176,4999,13177,13178,13179,13180,13181,13182,13183,13184,13185, //13168
911 13186,13187,13188,13189,13190,13191,13192,13193,13194,13195,13196,13197,13198,13199,13200,13201, //13184
912 13202,13203,13204,13205,13206,5000,13207,13208,13209,13210,13211,13212,13213,13214,13215,13216, //13200
913 13217,13218,13219,13220,13221,13222,13223,13224,13225,13226,13227,4200,5001,13228,13229,13230, //13216
914 13231,13232,13233,13234,13235,13236,13237,13238,13239,13240,3969,13241,13242,13243,13244,3970, //13232
915 13245,13246,13247,13248,13249,13250,13251,13252,13253,13254,13255,13256,13257,13258,13259,13260, //13248
916 13261,13262,13263,13264,13265,13266,13267,13268,3450,13269,13270,13271,13272,13273,13274,13275, //13264
917 13276,5002,13277,13278,13279,13280,13281,13282,13283,13284,13285,13286,13287,13288,13289,13290, //13280
918 13291,13292,13293,13294,13295,13296,13297,13298,13299,13300,13301,13302,3813,13303,13304,13305, //13296
919 13306,13307,13308,13309,13310,13311,13312,13313,13314,13315,13316,13317,13318,13319,13320,13321, //13312
920 13322,13323,13324,13325,13326,13327,13328,4507,13329,13330,13331,13332,13333,13334,13335,13336, //13328
921 13337,13338,13339,13340,13341,5003,13342,13343,13344,13345,13346,13347,13348,13349,13350,13351, //13344
922 13352,13353,13354,13355,13356,13357,13358,13359,13360,13361,13362,13363,13364,13365,13366,13367, //13360
923 5004,13368,13369,13370,13371,13372,13373,13374,13375,13376,13377,13378,13379,13380,13381,13382, //13376
924 13383,13384,13385,13386,13387,13388,13389,13390,13391,13392,13393,13394,13395,13396,13397,13398, //13392
925 13399,13400,13401,13402,13403,13404,13405,13406,13407,13408,13409,13410,13411,13412,13413,13414, //13408
926 13415,13416,13417,13418,13419,13420,13421,13422,13423,13424,13425,13426,13427,13428,13429,13430, //13424
927 13431,13432,4508,13433,13434,13435,4201,13436,13437,13438,13439,13440,13441,13442,13443,13444, //13440
928 13445,13446,13447,13448,13449,13450,13451,13452,13453,13454,13455,13456,13457,5005,13458,13459, //13456
929 13460,13461,13462,13463,13464,13465,13466,13467,13468,13469,13470,4509,13471,13472,13473,13474, //13472
930 13475,13476,13477,13478,13479,13480,13481,13482,13483,13484,13485,13486,13487,13488,13489,13490, //13488
931 13491,13492,13493,13494,13495,13496,13497,13498,13499,13500,13501,13502,13503,13504,13505,13506, //13504
932 13507,13508,13509,13510,13511,13512,13513,13514,13515,13516,13517,13518,13519,13520,13521,13522, //13520
933 13523,13524,13525,13526,13527,13528,13529,13530,13531,13532,13533,13534,13535,13536,13537,13538, //13536
934 13539,13540,13541,13542,13543,13544,13545,13546,13547,13548,13549,13550,13551,13552,13553,13554, //13552
935 13555,13556,13557,13558,13559,13560,13561,13562,13563,13564,13565,13566,13567,13568,13569,13570, //13568
936 13571,13572,13573,13574,13575,13576,13577,13578,13579,13580,13581,13582,13583,13584,13585,13586, //13584
937 13587,13588,13589,13590,13591,13592,13593,13594,13595,13596,13597,13598,13599,13600,13601,13602, //13600
938 13603,13604,13605,13606,13607,13608,13609,13610,13611,13612,13613,13614,13615,13616,13617,13618, //13616
939 13619,13620,13621,13622,13623,13624,13625,13626,13627,13628,13629,13630,13631,13632,13633,13634, //13632
940 13635,13636,13637,13638,13639,13640,13641,13642,5006,13643,13644,13645,13646,13647,13648,13649, //13648
941 13650,13651,5007,13652,13653,13654,13655,13656,13657,13658,13659,13660,13661,13662,13663,13664, //13664
942 13665,13666,13667,13668,13669,13670,13671,13672,13673,13674,13675,13676,13677,13678,13679,13680, //13680
943 13681,13682,13683,13684,13685,13686,13687,13688,13689,13690,13691,13692,13693,13694,13695,13696, //13696
944 13697,13698,13699,13700,13701,13702,13703,13704,13705,13706,13707,13708,13709,13710,13711,13712, //13712
945 13713,13714,13715,13716,13717,13718,13719,13720,13721,13722,13723,13724,13725,13726,13727,13728, //13728
946 13729,13730,13731,13732,13733,13734,13735,13736,13737,13738,13739,13740,13741,13742,13743,13744, //13744
947 13745,13746,13747,13748,13749,13750,13751,13752,13753,13754,13755,13756,13757,13758,13759,13760, //13760
948 13761,13762,13763,13764,13765,13766,13767,13768,13769,13770,13771,13772,13773,13774,3273,13775, //13776
949 13776,13777,13778,13779,13780,13781,13782,13783,13784,13785,13786,13787,13788,13789,13790,13791, //13792
950 13792,13793,13794,13795,13796,13797,13798,13799,13800,13801,13802,13803,13804,13805,13806,13807, //13808
951 13808,13809,13810,13811,13812,13813,13814,13815,13816,13817,13818,13819,13820,13821,13822,13823, //13824
952 13824,13825,13826,13827,13828,13829,13830,13831,13832,13833,13834,13835,13836,13837,13838,13839, //13840
953 13840,13841,13842,13843,13844,13845,13846,13847,13848,13849,13850,13851,13852,13853,13854,13855, //13856
954 13856,13857,13858,13859,13860,13861,13862,13863,13864,13865,13866,13867,13868,13869,13870,13871, //13872
955 13872,13873,13874,13875,13876,13877,13878,13879,13880,13881,13882,13883,13884,13885,13886,13887, //13888
956 13888,13889,13890,13891,13892,13893,13894,13895,13896,13897,13898,13899,13900,13901,13902,13903, //13904
957 13904,13905,13906,13907,13908,13909,13910,13911,13912,13913,13914,13915,13916,13917,13918,13919, //13920
958 13920,13921,13922,13923,13924,13925,13926,13927,13928,13929,13930,13931,13932,13933,13934,13935, //13936
959 13936,13937,13938,13939,13940,13941,13942,13943,13944,13945,13946,13947,13948,13949,13950,13951, //13952
960 13952,13953,13954,13955,13956,13957,13958,13959,13960,13961,13962,13963,13964,13965,13966,13967, //13968
961 13968,13969,13970,13971,13972, //13973
962 ****************************************************************************************/
963 };
964 }
+0
-124
src/org/mozilla/universalchardet/prober/distributionanalysis/CharDistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39
40 public abstract class CharDistributionAnalysis
41 {
42 ////////////////////////////////////////////////////////////////
43 // constants
44 ////////////////////////////////////////////////////////////////
45 public static final float SURE_NO = 0.01f;
46 public static final float SURE_YES = 0.99f;
47 public static final int ENOUGH_DATA_THRESHOLD = 1024;
48 public static final int MINIMUM_DATA_THRESHOLD = 4;
49
50
51 ////////////////////////////////////////////////////////////////
52 // fields
53 ////////////////////////////////////////////////////////////////
54 private int freqChars;
55 private int totalChars;
56 protected int[] charToFreqOrder; // set by subclasses
57 protected float typicalDistributionRatio; // set by subclasses
58 protected boolean done; // set by subclasses and reset()
59
60
61 ////////////////////////////////////////////////////////////////
62 // methods
63 ////////////////////////////////////////////////////////////////
64 public CharDistributionAnalysis()
65 {
66 reset();
67 }
68
69 public void handleData(final byte[] buf, int offset, int length)
70 {}
71
72 public void handleOneChar(final byte[] buf, int offset, int charLength)
73 {
74 int order = -1;
75
76 if (charLength == 2) {
77 order = getOrder(buf, offset);
78 }
79
80 if (order >= 0) {
81 ++this.totalChars;
82 if (order < this.charToFreqOrder.length) {
83 if (512 > this.charToFreqOrder[order]) {
84 ++this.freqChars;
85 }
86 }
87 }
88 }
89
90 public float getConfidence()
91 {
92 if (this.totalChars <= 0 || this.freqChars <= MINIMUM_DATA_THRESHOLD) {
93 return SURE_NO;
94 }
95
96 if (this.totalChars != this.freqChars) {
97 float r = this.freqChars / (this.totalChars - this.freqChars) * this.typicalDistributionRatio;
98
99 if (r < SURE_YES) {
100 return r;
101 }
102 }
103
104 return SURE_YES;
105 }
106
107 public void reset()
108 {
109 this.done = false;
110 this.totalChars = 0;
111 this.freqChars = 0;
112 }
113
114 public void setOption()
115 {}
116
117 public boolean gotEnoughData()
118 {
119 return (this.totalChars > ENOUGH_DATA_THRESHOLD);
120 }
121
122 protected abstract int getOrder(final byte[] buf, int offset);
123 }
+0
-70
src/org/mozilla/universalchardet/prober/distributionanalysis/EUCJPDistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public class EUCJPDistributionAnalysis extends JISDistributionAnalysis
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final int HIGHBYTE_BEGIN = 0xA1;
45 public static final int HIGHBYTE_END = 0xFE;
46 public static final int LOWBYTE_BEGIN = 0xA1;
47 public static final int LOWBYTE_END = 0xFE;
48
49
50 ////////////////////////////////////////////////////////////////
51 // methods
52 ////////////////////////////////////////////////////////////////
53 public EUCJPDistributionAnalysis()
54 {
55 super();
56 }
57
58 @Override
59 protected int getOrder(final byte[] buf, int offset)
60 {
61 int highbyte = buf[offset] & 0xFF;
62 if (highbyte >= HIGHBYTE_BEGIN) {
63 int lowbyte = buf[offset+1] & 0xFF;
64 return (94 * (highbyte - HIGHBYTE_BEGIN) + lowbyte - LOWBYTE_BEGIN);
65 } else {
66 return -1;
67 }
68 }
69 }
+0
-633
src/org/mozilla/universalchardet/prober/distributionanalysis/EUCKRDistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public class EUCKRDistributionAnalysis extends CharDistributionAnalysis
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final float EUCKR_TYPICAL_DISTRIBUTION_RATIO = 6.0f;
45 public static final int HIGHBYTE_BEGIN = 0xB0;
46 public static final int HIGHBYTE_END = 0xFE;
47 public static final int LOWBYTE_BEGIN = 0xA1;
48 public static final int LOWBYTE_END = 0xFE;
49
50
51 ////////////////////////////////////////////////////////////////
52 // methods
53 ////////////////////////////////////////////////////////////////
54 public EUCKRDistributionAnalysis()
55 {
56 super();
57 this.typicalDistributionRatio = EUCKR_TYPICAL_DISTRIBUTION_RATIO;
58 this.charToFreqOrder = euckrCharToFreqOrder;
59 }
60
61 @Override
62 protected int getOrder(final byte[] buf, int offset)
63 {
64 int highbyte = buf[offset] & 0xFF;
65 if (highbyte >= HIGHBYTE_BEGIN) {
66 int lowbyte = buf[offset+1] & 0xFF;
67 return (94 * (highbyte - HIGHBYTE_BEGIN) + lowbyte - LOWBYTE_BEGIN);
68 } else {
69 return -1;
70 }
71 }
72
73
74 ////////////////////////////////////////////////////////////////
75 // constants continued
76 ////////////////////////////////////////////////////////////////
77 private static final int[] euckrCharToFreqOrder = new int[] {
78 13, 130, 120,1396, 481,1719,1720, 328, 609, 212,1721, 707, 400, 299,1722, 87,
79 1397,1723, 104, 536,1117,1203,1724,1267, 685,1268, 508,1725,1726,1727,1728,1398,
80 1399,1729,1730,1731, 141, 621, 326,1057, 368,1732, 267, 488, 20,1733,1269,1734,
81 945,1400,1735, 47, 904,1270,1736,1737, 773, 248,1738, 409, 313, 786, 429,1739,
82 116, 987, 813,1401, 683, 75,1204, 145,1740,1741,1742,1743, 16, 847, 667, 622,
83 708,1744,1745,1746, 966, 787, 304, 129,1747, 60, 820, 123, 676,1748,1749,1750,
84 1751, 617,1752, 626,1753,1754,1755,1756, 653,1757,1758,1759,1760,1761,1762, 856,
85 344,1763,1764,1765,1766, 89, 401, 418, 806, 905, 848,1767,1768,1769, 946,1205,
86 709,1770,1118,1771, 241,1772,1773,1774,1271,1775, 569,1776, 999,1777,1778,1779,
87 1780, 337, 751,1058, 28, 628, 254,1781, 177, 906, 270, 349, 891,1079,1782, 19,
88 1783, 379,1784, 315,1785, 629, 754,1402, 559,1786, 636, 203,1206,1787, 710, 567,
89 1788, 935, 814,1789,1790,1207, 766, 528,1791,1792,1208,1793,1794,1795,1796,1797,
90 1403,1798,1799, 533,1059,1404,1405,1156,1406, 936, 884,1080,1800, 351,1801,1802,
91 1803,1804,1805, 801,1806,1807,1808,1119,1809,1157, 714, 474,1407,1810, 298, 899,
92 885,1811,1120, 802,1158,1812, 892,1813,1814,1408, 659,1815,1816,1121,1817,1818,
93 1819,1820,1821,1822, 319,1823, 594, 545,1824, 815, 937,1209,1825,1826, 573,1409,
94 1022,1827,1210,1828,1829,1830,1831,1832,1833, 556, 722, 807,1122,1060,1834, 697,
95 1835, 900, 557, 715,1836,1410, 540,1411, 752,1159, 294, 597,1211, 976, 803, 770,
96 1412,1837,1838, 39, 794,1413, 358,1839, 371, 925,1840, 453, 661, 788, 531, 723,
97 544,1023,1081, 869, 91,1841, 392, 430, 790, 602,1414, 677,1082, 457,1415,1416,
98 1842,1843, 475, 327,1024,1417, 795, 121,1844, 733, 403,1418,1845,1846,1847, 300,
99 119, 711,1212, 627,1848,1272, 207,1849,1850, 796,1213, 382,1851, 519,1852,1083,
100 893,1853,1854,1855, 367, 809, 487, 671,1856, 663,1857,1858, 956, 471, 306, 857,
101 1859,1860,1160,1084,1861,1862,1863,1864,1865,1061,1866,1867,1868,1869,1870,1871,
102 282, 96, 574,1872, 502,1085,1873,1214,1874, 907,1875,1876, 827, 977,1419,1420,
103 1421, 268,1877,1422,1878,1879,1880, 308,1881, 2, 537,1882,1883,1215,1884,1885,
104 127, 791,1886,1273,1423,1887, 34, 336, 404, 643,1888, 571, 654, 894, 840,1889,
105 0, 886,1274, 122, 575, 260, 908, 938,1890,1275, 410, 316,1891,1892, 100,1893,
106 1894,1123, 48,1161,1124,1025,1895, 633, 901,1276,1896,1897, 115, 816,1898, 317,
107 1899, 694,1900, 909, 734,1424, 572, 866,1425, 691, 85, 524,1010, 543, 394, 841,
108 1901,1902,1903,1026,1904,1905,1906,1907,1908,1909, 30, 451, 651, 988, 310,1910,
109 1911,1426, 810,1216, 93,1912,1913,1277,1217,1914, 858, 759, 45, 58, 181, 610,
110 269,1915,1916, 131,1062, 551, 443,1000, 821,1427, 957, 895,1086,1917,1918, 375,
111 1919, 359,1920, 687,1921, 822,1922, 293,1923,1924, 40, 662, 118, 692, 29, 939,
112 887, 640, 482, 174,1925, 69,1162, 728,1428, 910,1926,1278,1218,1279, 386, 870,
113 217, 854,1163, 823,1927,1928,1929,1930, 834,1931, 78,1932, 859,1933,1063,1934,
114 1935,1936,1937, 438,1164, 208, 595,1938,1939,1940,1941,1219,1125,1942, 280, 888,
115 1429,1430,1220,1431,1943,1944,1945,1946,1947,1280, 150, 510,1432,1948,1949,1950,
116 1951,1952,1953,1954,1011,1087,1955,1433,1043,1956, 881,1957, 614, 958,1064,1065,
117 1221,1958, 638,1001, 860, 967, 896,1434, 989, 492, 553,1281,1165,1959,1282,1002,
118 1283,1222,1960,1961,1962,1963, 36, 383, 228, 753, 247, 454,1964, 876, 678,1965,
119 1966,1284, 126, 464, 490, 835, 136, 672, 529, 940,1088,1435, 473,1967,1968, 467,
120 50, 390, 227, 587, 279, 378, 598, 792, 968, 240, 151, 160, 849, 882,1126,1285,
121 639,1044, 133, 140, 288, 360, 811, 563,1027, 561, 142, 523,1969,1970,1971, 7,
122 103, 296, 439, 407, 506, 634, 990,1972,1973,1974,1975, 645,1976,1977,1978,1979,
123 1980,1981, 236,1982,1436,1983,1984,1089, 192, 828, 618, 518,1166, 333,1127,1985,
124 818,1223,1986,1987,1988,1989,1990,1991,1992,1993, 342,1128,1286, 746, 842,1994,
125 1995, 560, 223,1287, 98, 8, 189, 650, 978,1288,1996,1437,1997, 17, 345, 250,
126 423, 277, 234, 512, 226, 97, 289, 42, 167,1998, 201,1999,2000, 843, 836, 824,
127 532, 338, 783,1090, 182, 576, 436,1438,1439, 527, 500,2001, 947, 889,2002,2003,
128 2004,2005, 262, 600, 314, 447,2006, 547,2007, 693, 738,1129,2008, 71,1440, 745,
129 619, 688,2009, 829,2010,2011, 147,2012, 33, 948,2013,2014, 74, 224,2015, 61,
130 191, 918, 399, 637,2016,1028,1130, 257, 902,2017,2018,2019,2020,2021,2022,2023,
131 2024,2025,2026, 837,2027,2028,2029,2030, 179, 874, 591, 52, 724, 246,2031,2032,
132 2033,2034,1167, 969,2035,1289, 630, 605, 911,1091,1168,2036,2037,2038,1441, 912,
133 2039, 623,2040,2041, 253,1169,1290,2042,1442, 146, 620, 611, 577, 433,2043,1224,
134 719,1170, 959, 440, 437, 534, 84, 388, 480,1131, 159, 220, 198, 679,2044,1012,
135 819,1066,1443, 113,1225, 194, 318,1003,1029,2045,2046,2047,2048,1067,2049,2050,
136 2051,2052,2053, 59, 913, 112,2054, 632,2055, 455, 144, 739,1291,2056, 273, 681,
137 499,2057, 448,2058,2059, 760,2060,2061, 970, 384, 169, 245,1132,2062,2063, 414,
138 1444,2064,2065, 41, 235,2066, 157, 252, 877, 568, 919, 789, 580,2067, 725,2068,
139 2069,1292,2070,2071,1445,2072,1446,2073,2074, 55, 588, 66,1447, 271,1092,2075,
140 1226,2076, 960,1013, 372,2077,2078,2079,2080,2081,1293,2082,2083,2084,2085, 850,
141 2086,2087,2088,2089,2090, 186,2091,1068, 180,2092,2093,2094, 109,1227, 522, 606,
142 2095, 867,1448,1093, 991,1171, 926, 353,1133,2096, 581,2097,2098,2099,1294,1449,
143 1450,2100, 596,1172,1014,1228,2101,1451,1295,1173,1229,2102,2103,1296,1134,1452,
144 949,1135,2104,2105,1094,1453,1454,1455,2106,1095,2107,2108,2109,2110,2111,2112,
145 2113,2114,2115,2116,2117, 804,2118,2119,1230,1231, 805,1456, 405,1136,2120,2121,
146 2122,2123,2124, 720, 701,1297, 992,1457, 927,1004,2125,2126,2127,2128,2129,2130,
147 22, 417,2131, 303,2132, 385,2133, 971, 520, 513,2134,1174, 73,1096, 231, 274,
148 962,1458, 673,2135,1459,2136, 152,1137,2137,2138,2139,2140,1005,1138,1460,1139,
149 2141,2142,2143,2144, 11, 374, 844,2145, 154,1232, 46,1461,2146, 838, 830, 721,
150 1233, 106,2147, 90, 428, 462, 578, 566,1175, 352,2148,2149, 538,1234, 124,1298,
151 2150,1462, 761, 565,2151, 686,2152, 649,2153, 72, 173,2154, 460, 415,2155,1463,
152 2156,1235, 305,2157,2158,2159,2160,2161,2162, 579,2163,2164,2165,2166,2167, 747,
153 2168,2169,2170,2171,1464, 669,2172,2173,2174,2175,2176,1465,2177, 23, 530, 285,
154 2178, 335, 729,2179, 397,2180,2181,2182,1030,2183,2184, 698,2185,2186, 325,2187,
155 2188, 369,2189, 799,1097,1015, 348,2190,1069, 680,2191, 851,1466,2192,2193, 10,
156 2194, 613, 424,2195, 979, 108, 449, 589, 27, 172, 81,1031, 80, 774, 281, 350,
157 1032, 525, 301, 582,1176,2196, 674,1045,2197,2198,1467, 730, 762,2199,2200,2201,
158 2202,1468,2203, 993,2204,2205, 266,1070, 963,1140,2206,2207,2208, 664,1098, 972,
159 2209,2210,2211,1177,1469,1470, 871,2212,2213,2214,2215,2216,1471,2217,2218,2219,
160 2220,2221,2222,2223,2224,2225,2226,2227,1472,1236,2228,2229,2230,2231,2232,2233,
161 2234,2235,1299,2236,2237, 200,2238, 477, 373,2239,2240, 731, 825, 777,2241,2242,
162 2243, 521, 486, 548,2244,2245,2246,1473,1300, 53, 549, 137, 875, 76, 158,2247,
163 1301,1474, 469, 396,1016, 278, 712,2248, 321, 442, 503, 767, 744, 941,1237,1178,
164 1475,2249, 82, 178,1141,1179, 973,2250,1302,2251, 297,2252,2253, 570,2254,2255,
165 2256, 18, 450, 206,2257, 290, 292,1142,2258, 511, 162, 99, 346, 164, 735,2259,
166 1476,1477, 4, 554, 343, 798,1099,2260,1100,2261, 43, 171,1303, 139, 215,2262,
167 2263, 717, 775,2264,1033, 322, 216,2265, 831,2266, 149,2267,1304,2268,2269, 702,
168 1238, 135, 845, 347, 309,2270, 484,2271, 878, 655, 238,1006,1478,2272, 67,2273,
169 295,2274,2275, 461,2276, 478, 942, 412,2277,1034,2278,2279,2280, 265,2281, 541,
170 2282,2283,2284,2285,2286, 70, 852,1071,2287,2288,2289,2290, 21, 56, 509, 117,
171 432,2291,2292, 331, 980, 552,1101, 148, 284, 105, 393,1180,1239, 755,2293, 187,
172 2294,1046,1479,2295, 340,2296, 63,1047, 230,2297,2298,1305, 763,1306, 101, 800,
173 808, 494,2299,2300,2301, 903,2302, 37,1072, 14, 5,2303, 79, 675,2304, 312,
174 2305,2306,2307,2308,2309,1480, 6,1307,2310,2311,2312, 1, 470, 35, 24, 229,
175 2313, 695, 210, 86, 778, 15, 784, 592, 779, 32, 77, 855, 964,2314, 259,2315,
176 501, 380,2316,2317, 83, 981, 153, 689,1308,1481,1482,1483,2318,2319, 716,1484,
177 2320,2321,2322,2323,2324,2325,1485,2326,2327, 128, 57, 68, 261,1048, 211, 170,
178 1240, 31,2328, 51, 435, 742,2329,2330,2331, 635,2332, 264, 456,2333,2334,2335,
179 425,2336,1486, 143, 507, 263, 943,2337, 363, 920,1487, 256,1488,1102, 243, 601,
180 1489,2338,2339,2340,2341,2342,2343,2344, 861,2345,2346,2347,2348,2349,2350, 395,
181 2351,1490,1491, 62, 535, 166, 225,2352,2353, 668, 419,1241, 138, 604, 928,2354,
182 1181,2355,1492,1493,2356,2357,2358,1143,2359, 696,2360, 387, 307,1309, 682, 476,
183 2361,2362, 332, 12, 222, 156,2363, 232,2364, 641, 276, 656, 517,1494,1495,1035,
184 416, 736,1496,2365,1017, 586,2366,2367,2368,1497,2369, 242,2370,2371,2372,1498,
185 2373, 965, 713,2374,2375,2376,2377, 740, 982,1499, 944,1500,1007,2378,2379,1310,
186 1501,2380,2381,2382, 785, 329,2383,2384,1502,2385,2386,2387, 932,2388,1503,2389,
187 2390,2391,2392,1242,2393,2394,2395,2396,2397, 994, 950,2398,2399,2400,2401,1504,
188 1311,2402,2403,2404,2405,1049, 749,2406,2407, 853, 718,1144,1312,2408,1182,1505,
189 2409,2410, 255, 516, 479, 564, 550, 214,1506,1507,1313, 413, 239, 444, 339,1145,
190 1036,1508,1509,1314,1037,1510,1315,2411,1511,2412,2413,2414, 176, 703, 497, 624,
191 593, 921, 302,2415, 341, 165,1103,1512,2416,1513,2417,2418,2419, 376,2420, 700,
192 2421,2422,2423, 258, 768,1316,2424,1183,2425, 995, 608,2426,2427,2428,2429, 221,
193 2430,2431,2432,2433,2434,2435,2436,2437, 195, 323, 726, 188, 897, 983,1317, 377,
194 644,1050, 879,2438, 452,2439,2440,2441,2442,2443,2444, 914,2445,2446,2447,2448,
195 915, 489,2449,1514,1184,2450,2451, 515, 64, 427, 495,2452, 583,2453, 483, 485,
196 1038, 562, 213,1515, 748, 666,2454,2455,2456,2457, 334,2458, 780, 996,1008, 705,
197 1243,2459,2460,2461,2462,2463, 114,2464, 493,1146, 366, 163,1516, 961,1104,2465,
198 291,2466,1318,1105,2467,1517, 365,2468, 355, 951,1244,2469,1319,2470, 631,2471,
199 2472, 218,1320, 364, 320, 756,1518,1519,1321,1520,1322,2473,2474,2475,2476, 997,
200 2477,2478,2479,2480, 665,1185,2481, 916,1521,2482,2483,2484, 584, 684,2485,2486,
201 797,2487,1051,1186,2488,2489,2490,1522,2491,2492, 370,2493,1039,1187, 65,2494,
202 434, 205, 463,1188,2495, 125, 812, 391, 402, 826, 699, 286, 398, 155, 781, 771,
203 585,2496, 590, 505,1073,2497, 599, 244, 219, 917,1018, 952, 646,1523,2498,1323,
204 2499,2500, 49, 984, 354, 741,2501, 625,2502,1324,2503,1019, 190, 357, 757, 491,
205 95, 782, 868,2504,2505,2506,2507,2508,2509, 134,1524,1074, 422,1525, 898,2510,
206 161,2511,2512,2513,2514, 769,2515,1526,2516,2517, 411,1325,2518, 472,1527,2519,
207 2520,2521,2522,2523,2524, 985,2525,2526,2527,2528,2529,2530, 764,2531,1245,2532,
208 2533, 25, 204, 311,2534, 496,2535,1052,2536,2537,2538,2539,2540,2541,2542, 199,
209 704, 504, 468, 758, 657,1528, 196, 44, 839,1246, 272, 750,2543, 765, 862,2544,
210 2545,1326,2546, 132, 615, 933,2547, 732,2548,2549,2550,1189,1529,2551, 283,1247,
211 1053, 607, 929,2552,2553,2554, 930, 183, 872, 616,1040,1147,2555,1148,1020, 441,
212 249,1075,2556,2557,2558, 466, 743,2559,2560,2561, 92, 514, 426, 420, 526,2562,
213 2563,2564,2565,2566,2567,2568, 185,2569,2570,2571,2572, 776,1530, 658,2573, 362,
214 2574, 361, 922,1076, 793,2575,2576,2577,2578,2579,2580,1531, 251,2581,2582,2583,
215 2584,1532, 54, 612, 237,1327,2585,2586, 275, 408, 647, 111,2587,1533,1106, 465,
216 3, 458, 9, 38,2588, 107, 110, 890, 209, 26, 737, 498,2589,1534,2590, 431,
217 202, 88,1535, 356, 287,1107, 660,1149,2591, 381,1536, 986,1150, 445,1248,1151,
218 974,2592,2593, 846,2594, 446, 953, 184,1249,1250, 727,2595, 923, 193, 883,2596,
219 2597,2598, 102, 324, 539, 817,2599, 421,1041,2600, 832,2601, 94, 175, 197, 406,
220 2602, 459,2603,2604,2605,2606,2607, 330, 555,2608,2609,2610, 706,1108, 389,2611,
221 2612,2613,2614, 233,2615, 833, 558, 931, 954,1251,2616,2617,1537, 546,2618,2619,
222 1009,2620,2621,2622,1538, 690,1328,2623, 955,2624,1539,2625,2626, 772,2627,2628,
223 2629,2630,2631, 924, 648, 863, 603,2632,2633, 934,1540, 864, 865,2634, 642,1042,
224 670,1190,2635,2636,2637,2638, 168,2639, 652, 873, 542,1054,1541,2640,2641,2642, //512, 256
225
226 /***************************************************************************************
227 *Everything below is of no interest for detection purpose *
228 ***************************************************************************************
229
230 2643,2644,2645,2646,2647,2648,2649,2650,2651,2652,2653,2654,2655,2656,2657,2658,
231 2659,2660,2661,2662,2663,2664,2665,2666,2667,2668,2669,2670,2671,2672,2673,2674,
232 2675,2676,2677,2678,2679,2680,2681,2682,2683,2684,2685,2686,2687,2688,2689,2690,
233 2691,2692,2693,2694,2695,2696,2697,2698,2699,1542, 880,2700,2701,2702,2703,2704,
234 2705,2706,2707,2708,2709,2710,2711,2712,2713,2714,2715,2716,2717,2718,2719,2720,
235 2721,2722,2723,2724,2725,1543,2726,2727,2728,2729,2730,2731,2732,1544,2733,2734,
236 2735,2736,2737,2738,2739,2740,2741,2742,2743,2744,2745,2746,2747,2748,2749,2750,
237 2751,2752,2753,2754,1545,2755,2756,2757,2758,2759,2760,2761,2762,2763,2764,2765,
238 2766,1546,2767,1547,2768,2769,2770,2771,2772,2773,2774,2775,2776,2777,2778,2779,
239 2780,2781,2782,2783,2784,2785,2786,1548,2787,2788,2789,1109,2790,2791,2792,2793,
240 2794,2795,2796,2797,2798,2799,2800,2801,2802,2803,2804,2805,2806,2807,2808,2809,
241 2810,2811,2812,1329,2813,2814,2815,2816,2817,2818,2819,2820,2821,2822,2823,2824,
242 2825,2826,2827,2828,2829,2830,2831,2832,2833,2834,2835,2836,2837,2838,2839,2840,
243 2841,2842,2843,2844,2845,2846,2847,2848,2849,2850,2851,2852,2853,2854,2855,2856,
244 1549,2857,2858,2859,2860,1550,2861,2862,1551,2863,2864,2865,2866,2867,2868,2869,
245 2870,2871,2872,2873,2874,1110,1330,2875,2876,2877,2878,2879,2880,2881,2882,2883,
246 2884,2885,2886,2887,2888,2889,2890,2891,2892,2893,2894,2895,2896,2897,2898,2899,
247 2900,2901,2902,2903,2904,2905,2906,2907,2908,2909,2910,2911,2912,2913,2914,2915,
248 2916,2917,2918,2919,2920,2921,2922,2923,2924,2925,2926,2927,2928,2929,2930,1331,
249 2931,2932,2933,2934,2935,2936,2937,2938,2939,2940,2941,2942,2943,1552,2944,2945,
250 2946,2947,2948,2949,2950,2951,2952,2953,2954,2955,2956,2957,2958,2959,2960,2961,
251 2962,2963,2964,1252,2965,2966,2967,2968,2969,2970,2971,2972,2973,2974,2975,2976,
252 2977,2978,2979,2980,2981,2982,2983,2984,2985,2986,2987,2988,2989,2990,2991,2992,
253 2993,2994,2995,2996,2997,2998,2999,3000,3001,3002,3003,3004,3005,3006,3007,3008,
254 3009,3010,3011,3012,1553,3013,3014,3015,3016,3017,1554,3018,1332,3019,3020,3021,
255 3022,3023,3024,3025,3026,3027,3028,3029,3030,3031,3032,3033,3034,3035,3036,3037,
256 3038,3039,3040,3041,3042,3043,3044,3045,3046,3047,3048,3049,3050,1555,3051,3052,
257 3053,1556,1557,3054,3055,3056,3057,3058,3059,3060,3061,3062,3063,3064,3065,3066,
258 3067,1558,3068,3069,3070,3071,3072,3073,3074,3075,3076,1559,3077,3078,3079,3080,
259 3081,3082,3083,1253,3084,3085,3086,3087,3088,3089,3090,3091,3092,3093,3094,3095,
260 3096,3097,3098,3099,3100,3101,3102,3103,3104,3105,3106,3107,3108,1152,3109,3110,
261 3111,3112,3113,1560,3114,3115,3116,3117,1111,3118,3119,3120,3121,3122,3123,3124,
262 3125,3126,3127,3128,3129,3130,3131,3132,3133,3134,3135,3136,3137,3138,3139,3140,
263 3141,3142,3143,3144,3145,3146,3147,3148,3149,3150,3151,3152,3153,3154,3155,3156,
264 3157,3158,3159,3160,3161,3162,3163,3164,3165,3166,3167,3168,3169,3170,3171,3172,
265 3173,3174,3175,3176,1333,3177,3178,3179,3180,3181,3182,3183,3184,3185,3186,3187,
266 3188,3189,1561,3190,3191,1334,3192,3193,3194,3195,3196,3197,3198,3199,3200,3201,
267 3202,3203,3204,3205,3206,3207,3208,3209,3210,3211,3212,3213,3214,3215,3216,3217,
268 3218,3219,3220,3221,3222,3223,3224,3225,3226,3227,3228,3229,3230,3231,3232,3233,
269 3234,1562,3235,3236,3237,3238,3239,3240,3241,3242,3243,3244,3245,3246,3247,3248,
270 3249,3250,3251,3252,3253,3254,3255,3256,3257,3258,3259,3260,3261,3262,3263,3264,
271 3265,3266,3267,3268,3269,3270,3271,3272,3273,3274,3275,3276,3277,1563,3278,3279,
272 3280,3281,3282,3283,3284,3285,3286,3287,3288,3289,3290,3291,3292,3293,3294,3295,
273 3296,3297,3298,3299,3300,3301,3302,3303,3304,3305,3306,3307,3308,3309,3310,3311,
274 3312,3313,3314,3315,3316,3317,3318,3319,3320,3321,3322,3323,3324,3325,3326,3327,
275 3328,3329,3330,3331,3332,3333,3334,3335,3336,3337,3338,3339,3340,3341,3342,3343,
276 3344,3345,3346,3347,3348,3349,3350,3351,3352,3353,3354,3355,3356,3357,3358,3359,
277 3360,3361,3362,3363,3364,1335,3365,3366,3367,3368,3369,3370,3371,3372,3373,3374,
278 3375,3376,3377,3378,3379,3380,3381,3382,3383,3384,3385,3386,3387,1336,3388,3389,
279 3390,3391,3392,3393,3394,3395,3396,3397,3398,3399,3400,3401,3402,3403,3404,3405,
280 3406,3407,3408,3409,3410,3411,3412,3413,3414,1337,3415,3416,3417,3418,3419,1338,
281 3420,3421,3422,1564,1565,3423,3424,3425,3426,3427,3428,3429,3430,3431,1254,3432,
282 3433,3434,1339,3435,3436,3437,3438,3439,1566,3440,3441,3442,3443,3444,3445,3446,
283 3447,3448,3449,3450,3451,3452,3453,3454,1255,3455,3456,3457,3458,3459,1567,1191,
284 3460,1568,1569,3461,3462,3463,1570,3464,3465,3466,3467,3468,1571,3469,3470,3471,
285 3472,3473,1572,3474,3475,3476,3477,3478,3479,3480,3481,3482,3483,3484,3485,3486,
286 1340,3487,3488,3489,3490,3491,3492,1021,3493,3494,3495,3496,3497,3498,1573,3499,
287 1341,3500,3501,3502,3503,3504,3505,3506,3507,3508,3509,3510,3511,1342,3512,3513,
288 3514,3515,3516,1574,1343,3517,3518,3519,1575,3520,1576,3521,3522,3523,3524,3525,
289 3526,3527,3528,3529,3530,3531,3532,3533,3534,3535,3536,3537,3538,3539,3540,3541,
290 3542,3543,3544,3545,3546,3547,3548,3549,3550,3551,3552,3553,3554,3555,3556,3557,
291 3558,3559,3560,3561,3562,3563,3564,3565,3566,3567,3568,3569,3570,3571,3572,3573,
292 3574,3575,3576,3577,3578,3579,3580,1577,3581,3582,1578,3583,3584,3585,3586,3587,
293 3588,3589,3590,3591,3592,3593,3594,3595,3596,3597,3598,3599,3600,3601,3602,3603,
294 3604,1579,3605,3606,3607,3608,3609,3610,3611,3612,3613,3614,3615,3616,3617,3618,
295 3619,3620,3621,3622,3623,3624,3625,3626,3627,3628,3629,1580,3630,3631,1581,3632,
296 3633,3634,3635,3636,3637,3638,3639,3640,3641,3642,3643,3644,3645,3646,3647,3648,
297 3649,3650,3651,3652,3653,3654,3655,3656,1582,3657,3658,3659,3660,3661,3662,3663,
298 3664,3665,3666,3667,3668,3669,3670,3671,3672,3673,3674,3675,3676,3677,3678,3679,
299 3680,3681,3682,3683,3684,3685,3686,3687,3688,3689,3690,3691,3692,3693,3694,3695,
300 3696,3697,3698,3699,3700,1192,3701,3702,3703,3704,1256,3705,3706,3707,3708,1583,
301 1257,3709,3710,3711,3712,3713,3714,3715,3716,1584,3717,3718,3719,3720,3721,3722,
302 3723,3724,3725,3726,3727,3728,3729,3730,3731,3732,3733,3734,3735,3736,3737,3738,
303 3739,3740,3741,3742,3743,3744,3745,1344,3746,3747,3748,3749,3750,3751,3752,3753,
304 3754,3755,3756,1585,3757,3758,3759,3760,3761,3762,3763,3764,3765,3766,1586,3767,
305 3768,3769,3770,3771,3772,3773,3774,3775,3776,3777,3778,1345,3779,3780,3781,3782,
306 3783,3784,3785,3786,3787,3788,3789,3790,3791,3792,3793,3794,3795,1346,1587,3796,
307 3797,1588,3798,3799,3800,3801,3802,3803,3804,3805,3806,1347,3807,3808,3809,3810,
308 3811,1589,3812,3813,3814,3815,3816,3817,3818,3819,3820,3821,1590,3822,3823,1591,
309 1348,3824,3825,3826,3827,3828,3829,3830,1592,3831,3832,1593,3833,3834,3835,3836,
310 3837,3838,3839,3840,3841,3842,3843,3844,1349,3845,3846,3847,3848,3849,3850,3851,
311 3852,3853,3854,3855,3856,3857,3858,1594,3859,3860,3861,3862,3863,3864,3865,3866,
312 3867,3868,3869,1595,3870,3871,3872,3873,1596,3874,3875,3876,3877,3878,3879,3880,
313 3881,3882,3883,3884,3885,3886,1597,3887,3888,3889,3890,3891,3892,3893,3894,3895,
314 1598,3896,3897,3898,1599,1600,3899,1350,3900,1351,3901,3902,1352,3903,3904,3905,
315 3906,3907,3908,3909,3910,3911,3912,3913,3914,3915,3916,3917,3918,3919,3920,3921,
316 3922,3923,3924,1258,3925,3926,3927,3928,3929,3930,3931,1193,3932,1601,3933,3934,
317 3935,3936,3937,3938,3939,3940,3941,3942,3943,1602,3944,3945,3946,3947,3948,1603,
318 3949,3950,3951,3952,3953,3954,3955,3956,3957,3958,3959,3960,3961,3962,3963,3964,
319 3965,1604,3966,3967,3968,3969,3970,3971,3972,3973,3974,3975,3976,3977,1353,3978,
320 3979,3980,3981,3982,3983,3984,3985,3986,3987,3988,3989,3990,3991,1354,3992,3993,
321 3994,3995,3996,3997,3998,3999,4000,4001,4002,4003,4004,4005,4006,4007,4008,4009,
322 4010,4011,4012,4013,4014,4015,4016,4017,4018,4019,4020,4021,4022,4023,1355,4024,
323 4025,4026,4027,4028,4029,4030,4031,4032,4033,4034,4035,4036,4037,4038,4039,4040,
324 1605,4041,4042,4043,4044,4045,4046,4047,4048,4049,4050,4051,4052,4053,4054,4055,
325 4056,4057,4058,4059,4060,1606,4061,4062,4063,4064,1607,4065,4066,4067,4068,4069,
326 4070,4071,4072,4073,4074,4075,4076,1194,4077,4078,1608,4079,4080,4081,4082,4083,
327 4084,4085,4086,4087,1609,4088,4089,4090,4091,4092,4093,4094,4095,4096,4097,4098,
328 4099,4100,4101,4102,4103,4104,4105,4106,4107,4108,1259,4109,4110,4111,4112,4113,
329 4114,4115,4116,4117,4118,4119,4120,4121,4122,4123,4124,1195,4125,4126,4127,1610,
330 4128,4129,4130,4131,4132,4133,4134,4135,4136,4137,1356,4138,4139,4140,4141,4142,
331 4143,4144,1611,4145,4146,4147,4148,4149,4150,4151,4152,4153,4154,4155,4156,4157,
332 4158,4159,4160,4161,4162,4163,4164,4165,4166,4167,4168,4169,4170,4171,4172,4173,
333 4174,4175,4176,4177,4178,4179,4180,4181,4182,4183,4184,4185,4186,4187,4188,4189,
334 4190,4191,4192,4193,4194,4195,4196,4197,4198,4199,4200,4201,4202,4203,4204,4205,
335 4206,4207,4208,4209,4210,4211,4212,4213,4214,4215,4216,4217,4218,4219,1612,4220,
336 4221,4222,4223,4224,4225,4226,4227,1357,4228,1613,4229,4230,4231,4232,4233,4234,
337 4235,4236,4237,4238,4239,4240,4241,4242,4243,1614,4244,4245,4246,4247,4248,4249,
338 4250,4251,4252,4253,4254,4255,4256,4257,4258,4259,4260,4261,4262,4263,4264,4265,
339 4266,4267,4268,4269,4270,1196,1358,4271,4272,4273,4274,4275,4276,4277,4278,4279,
340 4280,4281,4282,4283,4284,4285,4286,4287,1615,4288,4289,4290,4291,4292,4293,4294,
341 4295,4296,4297,4298,4299,4300,4301,4302,4303,4304,4305,4306,4307,4308,4309,4310,
342 4311,4312,4313,4314,4315,4316,4317,4318,4319,4320,4321,4322,4323,4324,4325,4326,
343 4327,4328,4329,4330,4331,4332,4333,4334,1616,4335,4336,4337,4338,4339,4340,4341,
344 4342,4343,4344,4345,4346,4347,4348,4349,4350,4351,4352,4353,4354,4355,4356,4357,
345 4358,4359,4360,1617,4361,4362,4363,4364,4365,1618,4366,4367,4368,4369,4370,4371,
346 4372,4373,4374,4375,4376,4377,4378,4379,4380,4381,4382,4383,4384,4385,4386,4387,
347 4388,4389,4390,4391,4392,4393,4394,4395,4396,4397,4398,4399,4400,4401,4402,4403,
348 4404,4405,4406,4407,4408,4409,4410,4411,4412,4413,4414,4415,4416,1619,4417,4418,
349 4419,4420,4421,4422,4423,4424,4425,1112,4426,4427,4428,4429,4430,1620,4431,4432,
350 4433,4434,4435,4436,4437,4438,4439,4440,4441,4442,1260,1261,4443,4444,4445,4446,
351 4447,4448,4449,4450,4451,4452,4453,4454,4455,1359,4456,4457,4458,4459,4460,4461,
352 4462,4463,4464,4465,1621,4466,4467,4468,4469,4470,4471,4472,4473,4474,4475,4476,
353 4477,4478,4479,4480,4481,4482,4483,4484,4485,4486,4487,4488,4489,1055,4490,4491,
354 4492,4493,4494,4495,4496,4497,4498,4499,4500,4501,4502,4503,4504,4505,4506,4507,
355 4508,4509,4510,4511,4512,4513,4514,4515,4516,4517,4518,1622,4519,4520,4521,1623,
356 4522,4523,4524,4525,4526,4527,4528,4529,4530,4531,4532,4533,4534,4535,1360,4536,
357 4537,4538,4539,4540,4541,4542,4543, 975,4544,4545,4546,4547,4548,4549,4550,4551,
358 4552,4553,4554,4555,4556,4557,4558,4559,4560,4561,4562,4563,4564,4565,4566,4567,
359 4568,4569,4570,4571,1624,4572,4573,4574,4575,4576,1625,4577,4578,4579,4580,4581,
360 4582,4583,4584,1626,4585,4586,4587,4588,4589,4590,4591,4592,4593,4594,4595,1627,
361 4596,4597,4598,4599,4600,4601,4602,4603,4604,4605,4606,4607,4608,4609,4610,4611,
362 4612,4613,4614,4615,1628,4616,4617,4618,4619,4620,4621,4622,4623,4624,4625,4626,
363 4627,4628,4629,4630,4631,4632,4633,4634,4635,4636,4637,4638,4639,4640,4641,4642,
364 4643,4644,4645,4646,4647,4648,4649,1361,4650,4651,4652,4653,4654,4655,4656,4657,
365 4658,4659,4660,4661,1362,4662,4663,4664,4665,4666,4667,4668,4669,4670,4671,4672,
366 4673,4674,4675,4676,4677,4678,4679,4680,4681,4682,1629,4683,4684,4685,4686,4687,
367 1630,4688,4689,4690,4691,1153,4692,4693,4694,1113,4695,4696,4697,4698,4699,4700,
368 4701,4702,4703,4704,4705,4706,4707,4708,4709,4710,4711,1197,4712,4713,4714,4715,
369 4716,4717,4718,4719,4720,4721,4722,4723,4724,4725,4726,4727,4728,4729,4730,4731,
370 4732,4733,4734,4735,1631,4736,1632,4737,4738,4739,4740,4741,4742,4743,4744,1633,
371 4745,4746,4747,4748,4749,1262,4750,4751,4752,4753,4754,1363,4755,4756,4757,4758,
372 4759,4760,4761,4762,4763,4764,4765,4766,4767,4768,1634,4769,4770,4771,4772,4773,
373 4774,4775,4776,4777,4778,1635,4779,4780,4781,4782,4783,4784,4785,4786,4787,4788,
374 4789,1636,4790,4791,4792,4793,4794,4795,4796,4797,4798,4799,4800,4801,4802,4803,
375 4804,4805,4806,1637,4807,4808,4809,1638,4810,4811,4812,4813,4814,4815,4816,4817,
376 4818,1639,4819,4820,4821,4822,4823,4824,4825,4826,4827,4828,4829,4830,4831,4832,
377 4833,1077,4834,4835,4836,4837,4838,4839,4840,4841,4842,4843,4844,4845,4846,4847,
378 4848,4849,4850,4851,4852,4853,4854,4855,4856,4857,4858,4859,4860,4861,4862,4863,
379 4864,4865,4866,4867,4868,4869,4870,4871,4872,4873,4874,4875,4876,4877,4878,4879,
380 4880,4881,4882,4883,1640,4884,4885,1641,4886,4887,4888,4889,4890,4891,4892,4893,
381 4894,4895,4896,4897,4898,4899,4900,4901,4902,4903,4904,4905,4906,4907,4908,4909,
382 4910,4911,1642,4912,4913,4914,1364,4915,4916,4917,4918,4919,4920,4921,4922,4923,
383 4924,4925,4926,4927,4928,4929,4930,4931,1643,4932,4933,4934,4935,4936,4937,4938,
384 4939,4940,4941,4942,4943,4944,4945,4946,4947,4948,4949,4950,4951,4952,4953,4954,
385 4955,4956,4957,4958,4959,4960,4961,4962,4963,4964,4965,4966,4967,4968,4969,4970,
386 4971,4972,4973,4974,4975,4976,4977,4978,4979,4980,1644,4981,4982,4983,4984,1645,
387 4985,4986,1646,4987,4988,4989,4990,4991,4992,4993,4994,4995,4996,4997,4998,4999,
388 5000,5001,5002,5003,5004,5005,1647,5006,1648,5007,5008,5009,5010,5011,5012,1078,
389 5013,5014,5015,5016,5017,5018,5019,5020,5021,5022,5023,5024,5025,5026,5027,5028,
390 1365,5029,5030,5031,5032,5033,5034,5035,5036,5037,5038,5039,1649,5040,5041,5042,
391 5043,5044,5045,1366,5046,5047,5048,5049,5050,5051,5052,5053,5054,5055,1650,5056,
392 5057,5058,5059,5060,5061,5062,5063,5064,5065,5066,5067,5068,5069,5070,5071,5072,
393 5073,5074,5075,5076,5077,1651,5078,5079,5080,5081,5082,5083,5084,5085,5086,5087,
394 5088,5089,5090,5091,5092,5093,5094,5095,5096,5097,5098,5099,5100,5101,5102,5103,
395 5104,5105,5106,5107,5108,5109,5110,1652,5111,5112,5113,5114,5115,5116,5117,5118,
396 1367,5119,5120,5121,5122,5123,5124,5125,5126,5127,5128,5129,1653,5130,5131,5132,
397 5133,5134,5135,5136,5137,5138,5139,5140,5141,5142,5143,5144,5145,5146,5147,5148,
398 5149,1368,5150,1654,5151,1369,5152,5153,5154,5155,5156,5157,5158,5159,5160,5161,
399 5162,5163,5164,5165,5166,5167,5168,5169,5170,5171,5172,5173,5174,5175,5176,5177,
400 5178,1370,5179,5180,5181,5182,5183,5184,5185,5186,5187,5188,5189,5190,5191,5192,
401 5193,5194,5195,5196,5197,5198,1655,5199,5200,5201,5202,1656,5203,5204,5205,5206,
402 1371,5207,1372,5208,5209,5210,5211,1373,5212,5213,1374,5214,5215,5216,5217,5218,
403 5219,5220,5221,5222,5223,5224,5225,5226,5227,5228,5229,5230,5231,5232,5233,5234,
404 5235,5236,5237,5238,5239,5240,5241,5242,5243,5244,5245,5246,5247,1657,5248,5249,
405 5250,5251,1658,1263,5252,5253,5254,5255,5256,1375,5257,5258,5259,5260,5261,5262,
406 5263,5264,5265,5266,5267,5268,5269,5270,5271,5272,5273,5274,5275,5276,5277,5278,
407 5279,5280,5281,5282,5283,1659,5284,5285,5286,5287,5288,5289,5290,5291,5292,5293,
408 5294,5295,5296,5297,5298,5299,5300,1660,5301,5302,5303,5304,5305,5306,5307,5308,
409 5309,5310,5311,5312,5313,5314,5315,5316,5317,5318,5319,5320,5321,1376,5322,5323,
410 5324,5325,5326,5327,5328,5329,5330,5331,5332,5333,1198,5334,5335,5336,5337,5338,
411 5339,5340,5341,5342,5343,1661,5344,5345,5346,5347,5348,5349,5350,5351,5352,5353,
412 5354,5355,5356,5357,5358,5359,5360,5361,5362,5363,5364,5365,5366,5367,5368,5369,
413 5370,5371,5372,5373,5374,5375,5376,5377,5378,5379,5380,5381,5382,5383,5384,5385,
414 5386,5387,5388,5389,5390,5391,5392,5393,5394,5395,5396,5397,5398,1264,5399,5400,
415 5401,5402,5403,5404,5405,5406,5407,5408,5409,5410,5411,5412,1662,5413,5414,5415,
416 5416,1663,5417,5418,5419,5420,5421,5422,5423,5424,5425,5426,5427,5428,5429,5430,
417 5431,5432,5433,5434,5435,5436,5437,5438,1664,5439,5440,5441,5442,5443,5444,5445,
418 5446,5447,5448,5449,5450,5451,5452,5453,5454,5455,5456,5457,5458,5459,5460,5461,
419 5462,5463,5464,5465,5466,5467,5468,5469,5470,5471,5472,5473,5474,5475,5476,5477,
420 5478,1154,5479,5480,5481,5482,5483,5484,5485,1665,5486,5487,5488,5489,5490,5491,
421 5492,5493,5494,5495,5496,5497,5498,5499,5500,5501,5502,5503,5504,5505,5506,5507,
422 5508,5509,5510,5511,5512,5513,5514,5515,5516,5517,5518,5519,5520,5521,5522,5523,
423 5524,5525,5526,5527,5528,5529,5530,5531,5532,5533,5534,5535,5536,5537,5538,5539,
424 5540,5541,5542,5543,5544,5545,5546,5547,5548,1377,5549,5550,5551,5552,5553,5554,
425 5555,5556,5557,5558,5559,5560,5561,5562,5563,5564,5565,5566,5567,5568,5569,5570,
426 1114,5571,5572,5573,5574,5575,5576,5577,5578,5579,5580,5581,5582,5583,5584,5585,
427 5586,5587,5588,5589,5590,5591,5592,1378,5593,5594,5595,5596,5597,5598,5599,5600,
428 5601,5602,5603,5604,5605,5606,5607,5608,5609,5610,5611,5612,5613,5614,1379,5615,
429 5616,5617,5618,5619,5620,5621,5622,5623,5624,5625,5626,5627,5628,5629,5630,5631,
430 5632,5633,5634,1380,5635,5636,5637,5638,5639,5640,5641,5642,5643,5644,5645,5646,
431 5647,5648,5649,1381,1056,5650,5651,5652,5653,5654,5655,5656,5657,5658,5659,5660,
432 1666,5661,5662,5663,5664,5665,5666,5667,5668,1667,5669,1668,5670,5671,5672,5673,
433 5674,5675,5676,5677,5678,1155,5679,5680,5681,5682,5683,5684,5685,5686,5687,5688,
434 5689,5690,5691,5692,5693,5694,5695,5696,5697,5698,1669,5699,5700,5701,5702,5703,
435 5704,5705,1670,5706,5707,5708,5709,5710,1671,5711,5712,5713,5714,1382,5715,5716,
436 5717,5718,5719,5720,5721,5722,5723,5724,5725,1672,5726,5727,1673,1674,5728,5729,
437 5730,5731,5732,5733,5734,5735,5736,1675,5737,5738,5739,5740,5741,5742,5743,5744,
438 1676,5745,5746,5747,5748,5749,5750,5751,1383,5752,5753,5754,5755,5756,5757,5758,
439 5759,5760,5761,5762,5763,5764,5765,5766,5767,5768,1677,5769,5770,5771,5772,5773,
440 1678,5774,5775,5776, 998,5777,5778,5779,5780,5781,5782,5783,5784,5785,1384,5786,
441 5787,5788,5789,5790,5791,5792,5793,5794,5795,5796,5797,5798,5799,5800,1679,5801,
442 5802,5803,1115,1116,5804,5805,5806,5807,5808,5809,5810,5811,5812,5813,5814,5815,
443 5816,5817,5818,5819,5820,5821,5822,5823,5824,5825,5826,5827,5828,5829,5830,5831,
444 5832,5833,5834,5835,5836,5837,5838,5839,5840,5841,5842,5843,5844,5845,5846,5847,
445 5848,5849,5850,5851,5852,5853,5854,5855,1680,5856,5857,5858,5859,5860,5861,5862,
446 5863,5864,1681,5865,5866,5867,1682,5868,5869,5870,5871,5872,5873,5874,5875,5876,
447 5877,5878,5879,1683,5880,1684,5881,5882,5883,5884,1685,5885,5886,5887,5888,5889,
448 5890,5891,5892,5893,5894,5895,5896,5897,5898,5899,5900,5901,5902,5903,5904,5905,
449 5906,5907,1686,5908,5909,5910,5911,5912,5913,5914,5915,5916,5917,5918,5919,5920,
450 5921,5922,5923,5924,5925,5926,5927,5928,5929,5930,5931,5932,5933,5934,5935,1687,
451 5936,5937,5938,5939,5940,5941,5942,5943,5944,5945,5946,5947,5948,5949,5950,5951,
452 5952,1688,1689,5953,1199,5954,5955,5956,5957,5958,5959,5960,5961,1690,5962,5963,
453 5964,5965,5966,5967,5968,5969,5970,5971,5972,5973,5974,5975,5976,5977,5978,5979,
454 5980,5981,1385,5982,1386,5983,5984,5985,5986,5987,5988,5989,5990,5991,5992,5993,
455 5994,5995,5996,5997,5998,5999,6000,6001,6002,6003,6004,6005,6006,6007,6008,6009,
456 6010,6011,6012,6013,6014,6015,6016,6017,6018,6019,6020,6021,6022,6023,6024,6025,
457 6026,6027,1265,6028,6029,1691,6030,6031,6032,6033,6034,6035,6036,6037,6038,6039,
458 6040,6041,6042,6043,6044,6045,6046,6047,6048,6049,6050,6051,6052,6053,6054,6055,
459 6056,6057,6058,6059,6060,6061,6062,6063,6064,6065,6066,6067,6068,6069,6070,6071,
460 6072,6073,6074,6075,6076,6077,6078,6079,6080,6081,6082,6083,6084,1692,6085,6086,
461 6087,6088,6089,6090,6091,6092,6093,6094,6095,6096,6097,6098,6099,6100,6101,6102,
462 6103,6104,6105,6106,6107,6108,6109,6110,6111,6112,6113,6114,6115,6116,6117,6118,
463 6119,6120,6121,6122,6123,6124,6125,6126,6127,6128,6129,6130,6131,1693,6132,6133,
464 6134,6135,6136,1694,6137,6138,6139,6140,6141,1695,6142,6143,6144,6145,6146,6147,
465 6148,6149,6150,6151,6152,6153,6154,6155,6156,6157,6158,6159,6160,6161,6162,6163,
466 6164,6165,6166,6167,6168,6169,6170,6171,6172,6173,6174,6175,6176,6177,6178,6179,
467 6180,6181,6182,6183,6184,6185,1696,6186,6187,6188,6189,6190,6191,6192,6193,6194,
468 6195,6196,6197,6198,6199,6200,6201,6202,6203,6204,6205,6206,6207,6208,6209,6210,
469 6211,6212,6213,6214,6215,6216,6217,6218,6219,1697,6220,6221,6222,6223,6224,6225,
470 6226,6227,6228,6229,6230,6231,6232,6233,6234,6235,6236,6237,6238,6239,6240,6241,
471 6242,6243,6244,6245,6246,6247,6248,6249,6250,6251,6252,6253,1698,6254,6255,6256,
472 6257,6258,6259,6260,6261,6262,6263,1200,6264,6265,6266,6267,6268,6269,6270,6271, //1024
473 6272,6273,6274,6275,6276,6277,6278,6279,6280,6281,6282,6283,6284,6285,6286,6287,
474 6288,6289,6290,6291,6292,6293,6294,6295,6296,6297,6298,6299,6300,6301,6302,1699,
475 6303,6304,1700,6305,6306,6307,6308,6309,6310,6311,6312,6313,6314,6315,6316,6317,
476 6318,6319,6320,6321,6322,6323,6324,6325,6326,6327,6328,6329,6330,6331,6332,6333,
477 6334,6335,6336,6337,6338,6339,1701,6340,6341,6342,6343,6344,1387,6345,6346,6347,
478 6348,6349,6350,6351,6352,6353,6354,6355,6356,6357,6358,6359,6360,6361,6362,6363,
479 6364,6365,6366,6367,6368,6369,6370,6371,6372,6373,6374,6375,6376,6377,6378,6379,
480 6380,6381,6382,6383,6384,6385,6386,6387,6388,6389,6390,6391,6392,6393,6394,6395,
481 6396,6397,6398,6399,6400,6401,6402,6403,6404,6405,6406,6407,6408,6409,6410,6411,
482 6412,6413,1702,6414,6415,6416,6417,6418,6419,6420,6421,6422,1703,6423,6424,6425,
483 6426,6427,6428,6429,6430,6431,6432,6433,6434,6435,6436,6437,6438,1704,6439,6440,
484 6441,6442,6443,6444,6445,6446,6447,6448,6449,6450,6451,6452,6453,6454,6455,6456,
485 6457,6458,6459,6460,6461,6462,6463,6464,6465,6466,6467,6468,6469,6470,6471,6472,
486 6473,6474,6475,6476,6477,6478,6479,6480,6481,6482,6483,6484,6485,6486,6487,6488,
487 6489,6490,6491,6492,6493,6494,6495,6496,6497,6498,6499,6500,6501,6502,6503,1266,
488 6504,6505,6506,6507,6508,6509,6510,6511,6512,6513,6514,6515,6516,6517,6518,6519,
489 6520,6521,6522,6523,6524,6525,6526,6527,6528,6529,6530,6531,6532,6533,6534,6535,
490 6536,6537,6538,6539,6540,6541,6542,6543,6544,6545,6546,6547,6548,6549,6550,6551,
491 1705,1706,6552,6553,6554,6555,6556,6557,6558,6559,6560,6561,6562,6563,6564,6565,
492 6566,6567,6568,6569,6570,6571,6572,6573,6574,6575,6576,6577,6578,6579,6580,6581,
493 6582,6583,6584,6585,6586,6587,6588,6589,6590,6591,6592,6593,6594,6595,6596,6597,
494 6598,6599,6600,6601,6602,6603,6604,6605,6606,6607,6608,6609,6610,6611,6612,6613,
495 6614,6615,6616,6617,6618,6619,6620,6621,6622,6623,6624,6625,6626,6627,6628,6629,
496 6630,6631,6632,6633,6634,6635,6636,6637,1388,6638,6639,6640,6641,6642,6643,6644,
497 1707,6645,6646,6647,6648,6649,6650,6651,6652,6653,6654,6655,6656,6657,6658,6659,
498 6660,6661,6662,6663,1708,6664,6665,6666,6667,6668,6669,6670,6671,6672,6673,6674,
499 1201,6675,6676,6677,6678,6679,6680,6681,6682,6683,6684,6685,6686,6687,6688,6689,
500 6690,6691,6692,6693,6694,6695,6696,6697,6698,6699,6700,6701,6702,6703,6704,6705,
501 6706,6707,6708,6709,6710,6711,6712,6713,6714,6715,6716,6717,6718,6719,6720,6721,
502 6722,6723,6724,6725,1389,6726,6727,6728,6729,6730,6731,6732,6733,6734,6735,6736,
503 1390,1709,6737,6738,6739,6740,6741,6742,1710,6743,6744,6745,6746,1391,6747,6748,
504 6749,6750,6751,6752,6753,6754,6755,6756,6757,1392,6758,6759,6760,6761,6762,6763,
505 6764,6765,6766,6767,6768,6769,6770,6771,6772,6773,6774,6775,6776,6777,6778,6779,
506 6780,1202,6781,6782,6783,6784,6785,6786,6787,6788,6789,6790,6791,6792,6793,6794,
507 6795,6796,6797,6798,6799,6800,6801,6802,6803,6804,6805,6806,6807,6808,6809,1711,
508 6810,6811,6812,6813,6814,6815,6816,6817,6818,6819,6820,6821,6822,6823,6824,6825,
509 6826,6827,6828,6829,6830,6831,6832,6833,6834,6835,6836,1393,6837,6838,6839,6840,
510 6841,6842,6843,6844,6845,6846,6847,6848,6849,6850,6851,6852,6853,6854,6855,6856,
511 6857,6858,6859,6860,6861,6862,6863,6864,6865,6866,6867,6868,6869,6870,6871,6872,
512 6873,6874,6875,6876,6877,6878,6879,6880,6881,6882,6883,6884,6885,6886,6887,6888,
513 6889,6890,6891,6892,6893,6894,6895,6896,6897,6898,6899,6900,6901,6902,1712,6903,
514 6904,6905,6906,6907,6908,6909,6910,1713,6911,6912,6913,6914,6915,6916,6917,6918,
515 6919,6920,6921,6922,6923,6924,6925,6926,6927,6928,6929,6930,6931,6932,6933,6934,
516 6935,6936,6937,6938,6939,6940,6941,6942,6943,6944,6945,6946,6947,6948,6949,6950,
517 6951,6952,6953,6954,6955,6956,6957,6958,6959,6960,6961,6962,6963,6964,6965,6966,
518 6967,6968,6969,6970,6971,6972,6973,6974,1714,6975,6976,6977,6978,6979,6980,6981,
519 6982,6983,6984,6985,6986,6987,6988,1394,6989,6990,6991,6992,6993,6994,6995,6996,
520 6997,6998,6999,7000,1715,7001,7002,7003,7004,7005,7006,7007,7008,7009,7010,7011,
521 7012,7013,7014,7015,7016,7017,7018,7019,7020,7021,7022,7023,7024,7025,7026,7027,
522 7028,1716,7029,7030,7031,7032,7033,7034,7035,7036,7037,7038,7039,7040,7041,7042,
523 7043,7044,7045,7046,7047,7048,7049,7050,7051,7052,7053,7054,7055,7056,7057,7058,
524 7059,7060,7061,7062,7063,7064,7065,7066,7067,7068,7069,7070,7071,7072,7073,7074,
525 7075,7076,7077,7078,7079,7080,7081,7082,7083,7084,7085,7086,7087,7088,7089,7090,
526 7091,7092,7093,7094,7095,7096,7097,7098,7099,7100,7101,7102,7103,7104,7105,7106,
527 7107,7108,7109,7110,7111,7112,7113,7114,7115,7116,7117,7118,7119,7120,7121,7122,
528 7123,7124,7125,7126,7127,7128,7129,7130,7131,7132,7133,7134,7135,7136,7137,7138,
529 7139,7140,7141,7142,7143,7144,7145,7146,7147,7148,7149,7150,7151,7152,7153,7154,
530 7155,7156,7157,7158,7159,7160,7161,7162,7163,7164,7165,7166,7167,7168,7169,7170,
531 7171,7172,7173,7174,7175,7176,7177,7178,7179,7180,7181,7182,7183,7184,7185,7186,
532 7187,7188,7189,7190,7191,7192,7193,7194,7195,7196,7197,7198,7199,7200,7201,7202,
533 7203,7204,7205,7206,7207,1395,7208,7209,7210,7211,7212,7213,1717,7214,7215,7216,
534 7217,7218,7219,7220,7221,7222,7223,7224,7225,7226,7227,7228,7229,7230,7231,7232,
535 7233,7234,7235,7236,7237,7238,7239,7240,7241,7242,7243,7244,7245,7246,7247,7248,
536 7249,7250,7251,7252,7253,7254,7255,7256,7257,7258,7259,7260,7261,7262,7263,7264,
537 7265,7266,7267,7268,7269,7270,7271,7272,7273,7274,7275,7276,7277,7278,7279,7280,
538 7281,7282,7283,7284,7285,7286,7287,7288,7289,7290,7291,7292,7293,7294,7295,7296,
539 7297,7298,7299,7300,7301,7302,7303,7304,7305,7306,7307,7308,7309,7310,7311,7312,
540 7313,1718,7314,7315,7316,7317,7318,7319,7320,7321,7322,7323,7324,7325,7326,7327,
541 7328,7329,7330,7331,7332,7333,7334,7335,7336,7337,7338,7339,7340,7341,7342,7343,
542 7344,7345,7346,7347,7348,7349,7350,7351,7352,7353,7354,7355,7356,7357,7358,7359,
543 7360,7361,7362,7363,7364,7365,7366,7367,7368,7369,7370,7371,7372,7373,7374,7375,
544 7376,7377,7378,7379,7380,7381,7382,7383,7384,7385,7386,7387,7388,7389,7390,7391,
545 7392,7393,7394,7395,7396,7397,7398,7399,7400,7401,7402,7403,7404,7405,7406,7407,
546 7408,7409,7410,7411,7412,7413,7414,7415,7416,7417,7418,7419,7420,7421,7422,7423,
547 7424,7425,7426,7427,7428,7429,7430,7431,7432,7433,7434,7435,7436,7437,7438,7439,
548 7440,7441,7442,7443,7444,7445,7446,7447,7448,7449,7450,7451,7452,7453,7454,7455,
549 7456,7457,7458,7459,7460,7461,7462,7463,7464,7465,7466,7467,7468,7469,7470,7471,
550 7472,7473,7474,7475,7476,7477,7478,7479,7480,7481,7482,7483,7484,7485,7486,7487,
551 7488,7489,7490,7491,7492,7493,7494,7495,7496,7497,7498,7499,7500,7501,7502,7503,
552 7504,7505,7506,7507,7508,7509,7510,7511,7512,7513,7514,7515,7516,7517,7518,7519,
553 7520,7521,7522,7523,7524,7525,7526,7527,7528,7529,7530,7531,7532,7533,7534,7535,
554 7536,7537,7538,7539,7540,7541,7542,7543,7544,7545,7546,7547,7548,7549,7550,7551,
555 7552,7553,7554,7555,7556,7557,7558,7559,7560,7561,7562,7563,7564,7565,7566,7567,
556 7568,7569,7570,7571,7572,7573,7574,7575,7576,7577,7578,7579,7580,7581,7582,7583,
557 7584,7585,7586,7587,7588,7589,7590,7591,7592,7593,7594,7595,7596,7597,7598,7599,
558 7600,7601,7602,7603,7604,7605,7606,7607,7608,7609,7610,7611,7612,7613,7614,7615,
559 7616,7617,7618,7619,7620,7621,7622,7623,7624,7625,7626,7627,7628,7629,7630,7631,
560 7632,7633,7634,7635,7636,7637,7638,7639,7640,7641,7642,7643,7644,7645,7646,7647,
561 7648,7649,7650,7651,7652,7653,7654,7655,7656,7657,7658,7659,7660,7661,7662,7663,
562 7664,7665,7666,7667,7668,7669,7670,7671,7672,7673,7674,7675,7676,7677,7678,7679,
563 7680,7681,7682,7683,7684,7685,7686,7687,7688,7689,7690,7691,7692,7693,7694,7695,
564 7696,7697,7698,7699,7700,7701,7702,7703,7704,7705,7706,7707,7708,7709,7710,7711,
565 7712,7713,7714,7715,7716,7717,7718,7719,7720,7721,7722,7723,7724,7725,7726,7727,
566 7728,7729,7730,7731,7732,7733,7734,7735,7736,7737,7738,7739,7740,7741,7742,7743,
567 7744,7745,7746,7747,7748,7749,7750,7751,7752,7753,7754,7755,7756,7757,7758,7759,
568 7760,7761,7762,7763,7764,7765,7766,7767,7768,7769,7770,7771,7772,7773,7774,7775,
569 7776,7777,7778,7779,7780,7781,7782,7783,7784,7785,7786,7787,7788,7789,7790,7791,
570 7792,7793,7794,7795,7796,7797,7798,7799,7800,7801,7802,7803,7804,7805,7806,7807,
571 7808,7809,7810,7811,7812,7813,7814,7815,7816,7817,7818,7819,7820,7821,7822,7823,
572 7824,7825,7826,7827,7828,7829,7830,7831,7832,7833,7834,7835,7836,7837,7838,7839,
573 7840,7841,7842,7843,7844,7845,7846,7847,7848,7849,7850,7851,7852,7853,7854,7855,
574 7856,7857,7858,7859,7860,7861,7862,7863,7864,7865,7866,7867,7868,7869,7870,7871,
575 7872,7873,7874,7875,7876,7877,7878,7879,7880,7881,7882,7883,7884,7885,7886,7887,
576 7888,7889,7890,7891,7892,7893,7894,7895,7896,7897,7898,7899,7900,7901,7902,7903,
577 7904,7905,7906,7907,7908,7909,7910,7911,7912,7913,7914,7915,7916,7917,7918,7919,
578 7920,7921,7922,7923,7924,7925,7926,7927,7928,7929,7930,7931,7932,7933,7934,7935,
579 7936,7937,7938,7939,7940,7941,7942,7943,7944,7945,7946,7947,7948,7949,7950,7951,
580 7952,7953,7954,7955,7956,7957,7958,7959,7960,7961,7962,7963,7964,7965,7966,7967,
581 7968,7969,7970,7971,7972,7973,7974,7975,7976,7977,7978,7979,7980,7981,7982,7983,
582 7984,7985,7986,7987,7988,7989,7990,7991,7992,7993,7994,7995,7996,7997,7998,7999,
583 8000,8001,8002,8003,8004,8005,8006,8007,8008,8009,8010,8011,8012,8013,8014,8015,
584 8016,8017,8018,8019,8020,8021,8022,8023,8024,8025,8026,8027,8028,8029,8030,8031,
585 8032,8033,8034,8035,8036,8037,8038,8039,8040,8041,8042,8043,8044,8045,8046,8047,
586 8048,8049,8050,8051,8052,8053,8054,8055,8056,8057,8058,8059,8060,8061,8062,8063,
587 8064,8065,8066,8067,8068,8069,8070,8071,8072,8073,8074,8075,8076,8077,8078,8079,
588 8080,8081,8082,8083,8084,8085,8086,8087,8088,8089,8090,8091,8092,8093,8094,8095,
589 8096,8097,8098,8099,8100,8101,8102,8103,8104,8105,8106,8107,8108,8109,8110,8111,
590 8112,8113,8114,8115,8116,8117,8118,8119,8120,8121,8122,8123,8124,8125,8126,8127,
591 8128,8129,8130,8131,8132,8133,8134,8135,8136,8137,8138,8139,8140,8141,8142,8143,
592 8144,8145,8146,8147,8148,8149,8150,8151,8152,8153,8154,8155,8156,8157,8158,8159,
593 8160,8161,8162,8163,8164,8165,8166,8167,8168,8169,8170,8171,8172,8173,8174,8175,
594 8176,8177,8178,8179,8180,8181,8182,8183,8184,8185,8186,8187,8188,8189,8190,8191,
595 8192,8193,8194,8195,8196,8197,8198,8199,8200,8201,8202,8203,8204,8205,8206,8207,
596 8208,8209,8210,8211,8212,8213,8214,8215,8216,8217,8218,8219,8220,8221,8222,8223,
597 8224,8225,8226,8227,8228,8229,8230,8231,8232,8233,8234,8235,8236,8237,8238,8239,
598 8240,8241,8242,8243,8244,8245,8246,8247,8248,8249,8250,8251,8252,8253,8254,8255,
599 8256,8257,8258,8259,8260,8261,8262,8263,8264,8265,8266,8267,8268,8269,8270,8271,
600 8272,8273,8274,8275,8276,8277,8278,8279,8280,8281,8282,8283,8284,8285,8286,8287,
601 8288,8289,8290,8291,8292,8293,8294,8295,8296,8297,8298,8299,8300,8301,8302,8303,
602 8304,8305,8306,8307,8308,8309,8310,8311,8312,8313,8314,8315,8316,8317,8318,8319,
603 8320,8321,8322,8323,8324,8325,8326,8327,8328,8329,8330,8331,8332,8333,8334,8335,
604 8336,8337,8338,8339,8340,8341,8342,8343,8344,8345,8346,8347,8348,8349,8350,8351,
605 8352,8353,8354,8355,8356,8357,8358,8359,8360,8361,8362,8363,8364,8365,8366,8367,
606 8368,8369,8370,8371,8372,8373,8374,8375,8376,8377,8378,8379,8380,8381,8382,8383,
607 8384,8385,8386,8387,8388,8389,8390,8391,8392,8393,8394,8395,8396,8397,8398,8399,
608 8400,8401,8402,8403,8404,8405,8406,8407,8408,8409,8410,8411,8412,8413,8414,8415,
609 8416,8417,8418,8419,8420,8421,8422,8423,8424,8425,8426,8427,8428,8429,8430,8431,
610 8432,8433,8434,8435,8436,8437,8438,8439,8440,8441,8442,8443,8444,8445,8446,8447,
611 8448,8449,8450,8451,8452,8453,8454,8455,8456,8457,8458,8459,8460,8461,8462,8463,
612 8464,8465,8466,8467,8468,8469,8470,8471,8472,8473,8474,8475,8476,8477,8478,8479,
613 8480,8481,8482,8483,8484,8485,8486,8487,8488,8489,8490,8491,8492,8493,8494,8495,
614 8496,8497,8498,8499,8500,8501,8502,8503,8504,8505,8506,8507,8508,8509,8510,8511,
615 8512,8513,8514,8515,8516,8517,8518,8519,8520,8521,8522,8523,8524,8525,8526,8527,
616 8528,8529,8530,8531,8532,8533,8534,8535,8536,8537,8538,8539,8540,8541,8542,8543,
617 8544,8545,8546,8547,8548,8549,8550,8551,8552,8553,8554,8555,8556,8557,8558,8559,
618 8560,8561,8562,8563,8564,8565,8566,8567,8568,8569,8570,8571,8572,8573,8574,8575,
619 8576,8577,8578,8579,8580,8581,8582,8583,8584,8585,8586,8587,8588,8589,8590,8591,
620 8592,8593,8594,8595,8596,8597,8598,8599,8600,8601,8602,8603,8604,8605,8606,8607,
621 8608,8609,8610,8611,8612,8613,8614,8615,8616,8617,8618,8619,8620,8621,8622,8623,
622 8624,8625,8626,8627,8628,8629,8630,8631,8632,8633,8634,8635,8636,8637,8638,8639,
623 8640,8641,8642,8643,8644,8645,8646,8647,8648,8649,8650,8651,8652,8653,8654,8655,
624 8656,8657,8658,8659,8660,8661,8662,8663,8664,8665,8666,8667,8668,8669,8670,8671,
625 8672,8673,8674,8675,8676,8677,8678,8679,8680,8681,8682,8683,8684,8685,8686,8687,
626 8688,8689,8690,8691,8692,8693,8694,8695,8696,8697,8698,8699,8700,8701,8702,8703,
627 8704,8705,8706,8707,8708,8709,8710,8711,8712,8713,8714,8715,8716,8717,8718,8719,
628 8720,8721,8722,8723,8724,8725,8726,8727,8728,8729,8730,8731,8732,8733,8734,8735,
629 8736,8737,8738,8739,8740,8741
630 ****************************************************************************************/
631 };
632 }
+0
-462
src/org/mozilla/universalchardet/prober/distributionanalysis/EUCTWDistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public class EUCTWDistributionAnalysis extends CharDistributionAnalysis
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final float EUCTW_TYPICAL_DISTRIBUTION_RATIO = 0.75f;
45 public static final int HIGHBYTE_BEGIN = 0xC4;
46 public static final int HIGHBYTE_END = 0xFE;
47 public static final int LOWBYTE_BEGIN = 0xA1;
48 public static final int LOWBYTE_END = 0xFE;
49
50
51 ////////////////////////////////////////////////////////////////
52 // methods
53 ////////////////////////////////////////////////////////////////
54 public EUCTWDistributionAnalysis()
55 {
56 super();
57 this.charToFreqOrder = euctwCharToFreqOrder;
58 this.typicalDistributionRatio = EUCTW_TYPICAL_DISTRIBUTION_RATIO;
59 }
60
61 @Override
62 protected int getOrder(final byte[] buf, int offset)
63 {
64 int highbyte = buf[offset] & 0xFF;
65 if (highbyte >= HIGHBYTE_BEGIN) {
66 int lowbyte = buf[offset+1] & 0xFF;
67 return (94 * (highbyte - HIGHBYTE_BEGIN) + lowbyte - LOWBYTE_BEGIN);
68 } else {
69 return -1;
70 }
71 }
72
73
74 ////////////////////////////////////////////////////////////////
75 // constants continued
76 ////////////////////////////////////////////////////////////////
77 private static final int[] euctwCharToFreqOrder = new int[] {
78 1,1800,1506, 255,1431, 198, 9, 82, 6,7310, 177, 202,3615,1256,2808, 110, // 2742
79 3735, 33,3241, 261, 76, 44,2113, 16,2931,2184,1176, 659,3868, 26,3404,2643, // 2758
80 1198,3869,3313,4060, 410,2211, 302, 590, 361,1963, 8, 204, 58,4296,7311,1931, // 2774
81 63,7312,7313, 317,1614, 75, 222, 159,4061,2412,1480,7314,3500,3068, 224,2809, // 2790
82 3616, 3, 10,3870,1471, 29,2774,1135,2852,1939, 873, 130,3242,1123, 312,7315, // 2806
83 4297,2051, 507, 252, 682,7316, 142,1914, 124, 206,2932, 34,3501,3173, 64, 604, // 2822
84 7317,2494,1976,1977, 155,1990, 645, 641,1606,7318,3405, 337, 72, 406,7319, 80, // 2838
85 630, 238,3174,1509, 263, 939,1092,2644, 756,1440,1094,3406, 449, 69,2969, 591, // 2854
86 179,2095, 471, 115,2034,1843, 60, 50,2970, 134, 806,1868, 734,2035,3407, 180, // 2870
87 995,1607, 156, 537,2893, 688,7320, 319,1305, 779,2144, 514,2374, 298,4298, 359, // 2886
88 2495, 90,2707,1338, 663, 11, 906,1099,2545, 20,2436, 182, 532,1716,7321, 732, // 2902
89 1376,4062,1311,1420,3175, 25,2312,1056, 113, 399, 382,1949, 242,3408,2467, 529, // 2918
90 3243, 475,1447,3617,7322, 117, 21, 656, 810,1297,2295,2329,3502,7323, 126,4063, // 2934
91 706, 456, 150, 613,4299, 71,1118,2036,4064, 145,3069, 85, 835, 486,2114,1246, // 2950
92 1426, 428, 727,1285,1015, 800, 106, 623, 303,1281,7324,2127,2354, 347,3736, 221, // 2966
93 3503,3110,7325,1955,1153,4065, 83, 296,1199,3070, 192, 624, 93,7326, 822,1897, // 2982
94 2810,3111, 795,2064, 991,1554,1542,1592, 27, 43,2853, 859, 139,1456, 860,4300, // 2998
95 437, 712,3871, 164,2392,3112, 695, 211,3017,2096, 195,3872,1608,3504,3505,3618, // 3014
96 3873, 234, 811,2971,2097,3874,2229,1441,3506,1615,2375, 668,2076,1638, 305, 228, // 3030
97 1664,4301, 467, 415,7327, 262,2098,1593, 239, 108, 300, 200,1033, 512,1247,2077, // 3046
98 7328,7329,2173,3176,3619,2673, 593, 845,1062,3244, 88,1723,2037,3875,1950, 212, // 3062
99 266, 152, 149, 468,1898,4066,4302, 77, 187,7330,3018, 37, 5,2972,7331,3876, // 3078
100 7332,7333, 39,2517,4303,2894,3177,2078, 55, 148, 74,4304, 545, 483,1474,1029, // 3094
101 1665, 217,1869,1531,3113,1104,2645,4067, 24, 172,3507, 900,3877,3508,3509,4305, // 3110
102 32,1408,2811,1312, 329, 487,2355,2247,2708, 784,2674, 4,3019,3314,1427,1788, // 3126
103 188, 109, 499,7334,3620,1717,1789, 888,1217,3020,4306,7335,3510,7336,3315,1520, // 3142
104 3621,3878, 196,1034, 775,7337,7338, 929,1815, 249, 439, 38,7339,1063,7340, 794, // 3158
105 3879,1435,2296, 46, 178,3245,2065,7341,2376,7342, 214,1709,4307, 804, 35, 707, // 3174
106 324,3622,1601,2546, 140, 459,4068,7343,7344,1365, 839, 272, 978,2257,2572,3409, // 3190
107 2128,1363,3623,1423, 697, 100,3071, 48, 70,1231, 495,3114,2193,7345,1294,7346, // 3206
108 2079, 462, 586,1042,3246, 853, 256, 988, 185,2377,3410,1698, 434,1084,7347,3411, // 3222
109 314,2615,2775,4308,2330,2331, 569,2280, 637,1816,2518, 757,1162,1878,1616,3412, // 3238
110 287,1577,2115, 768,4309,1671,2854,3511,2519,1321,3737, 909,2413,7348,4069, 933, // 3254
111 3738,7349,2052,2356,1222,4310, 765,2414,1322, 786,4311,7350,1919,1462,1677,2895, // 3270
112 1699,7351,4312,1424,2437,3115,3624,2590,3316,1774,1940,3413,3880,4070, 309,1369, // 3286
113 1130,2812, 364,2230,1653,1299,3881,3512,3882,3883,2646, 525,1085,3021, 902,2000, // 3302
114 1475, 964,4313, 421,1844,1415,1057,2281, 940,1364,3116, 376,4314,4315,1381, 7, // 3318
115 2520, 983,2378, 336,1710,2675,1845, 321,3414, 559,1131,3022,2742,1808,1132,1313, // 3334
116 265,1481,1857,7352, 352,1203,2813,3247, 167,1089, 420,2814, 776, 792,1724,3513, // 3350
117 4071,2438,3248,7353,4072,7354, 446, 229, 333,2743, 901,3739,1200,1557,4316,2647, // 3366
118 1920, 395,2744,2676,3740,4073,1835, 125, 916,3178,2616,4317,7355,7356,3741,7357, // 3382
119 7358,7359,4318,3117,3625,1133,2547,1757,3415,1510,2313,1409,3514,7360,2145, 438, // 3398
120 2591,2896,2379,3317,1068, 958,3023, 461, 311,2855,2677,4074,1915,3179,4075,1978, // 3414
121 383, 750,2745,2617,4076, 274, 539, 385,1278,1442,7361,1154,1964, 384, 561, 210, // 3430
122 98,1295,2548,3515,7362,1711,2415,1482,3416,3884,2897,1257, 129,7363,3742, 642, // 3446
123 523,2776,2777,2648,7364, 141,2231,1333, 68, 176, 441, 876, 907,4077, 603,2592, // 3462
124 710, 171,3417, 404, 549, 18,3118,2393,1410,3626,1666,7365,3516,4319,2898,4320, // 3478
125 7366,2973, 368,7367, 146, 366, 99, 871,3627,1543, 748, 807,1586,1185, 22,2258, // 3494
126 379,3743,3180,7368,3181, 505,1941,2618,1991,1382,2314,7369, 380,2357, 218, 702, // 3510
127 1817,1248,3418,3024,3517,3318,3249,7370,2974,3628, 930,3250,3744,7371, 59,7372, // 3526
128 585, 601,4078, 497,3419,1112,1314,4321,1801,7373,1223,1472,2174,7374, 749,1836, // 3542
129 690,1899,3745,1772,3885,1476, 429,1043,1790,2232,2116, 917,4079, 447,1086,1629, // 3558
130 7375, 556,7376,7377,2020,1654, 844,1090, 105, 550, 966,1758,2815,1008,1782, 686, // 3574
131 1095,7378,2282, 793,1602,7379,3518,2593,4322,4080,2933,2297,4323,3746, 980,2496, // 3590
132 544, 353, 527,4324, 908,2678,2899,7380, 381,2619,1942,1348,7381,1341,1252, 560, // 3606
133 3072,7382,3420,2856,7383,2053, 973, 886,2080, 143,4325,7384,7385, 157,3886, 496, // 3622
134 4081, 57, 840, 540,2038,4326,4327,3421,2117,1445, 970,2259,1748,1965,2081,4082, // 3638
135 3119,1234,1775,3251,2816,3629, 773,1206,2129,1066,2039,1326,3887,1738,1725,4083, // 3654
136 279,3120, 51,1544,2594, 423,1578,2130,2066, 173,4328,1879,7386,7387,1583, 264, // 3670
137 610,3630,4329,2439, 280, 154,7388,7389,7390,1739, 338,1282,3073, 693,2857,1411, // 3686
138 1074,3747,2440,7391,4330,7392,7393,1240, 952,2394,7394,2900,1538,2679, 685,1483, // 3702
139 4084,2468,1436, 953,4085,2054,4331, 671,2395, 79,4086,2441,3252, 608, 567,2680, // 3718
140 3422,4087,4088,1691, 393,1261,1791,2396,7395,4332,7396,7397,7398,7399,1383,1672, // 3734
141 3748,3182,1464, 522,1119, 661,1150, 216, 675,4333,3888,1432,3519, 609,4334,2681, // 3750
142 2397,7400,7401,7402,4089,3025, 0,7403,2469, 315, 231,2442, 301,3319,4335,2380, // 3766
143 7404, 233,4090,3631,1818,4336,4337,7405, 96,1776,1315,2082,7406, 257,7407,1809, // 3782
144 3632,2709,1139,1819,4091,2021,1124,2163,2778,1777,2649,7408,3074, 363,1655,3183, // 3798
145 7409,2975,7410,7411,7412,3889,1567,3890, 718, 103,3184, 849,1443, 341,3320,2934, // 3814
146 1484,7413,1712, 127, 67, 339,4092,2398, 679,1412, 821,7414,7415, 834, 738, 351, // 3830
147 2976,2146, 846, 235,1497,1880, 418,1992,3749,2710, 186,1100,2147,2746,3520,1545, // 3846
148 1355,2935,2858,1377, 583,3891,4093,2573,2977,7416,1298,3633,1078,2549,3634,2358, // 3862
149 78,3750,3751, 267,1289,2099,2001,1594,4094, 348, 369,1274,2194,2175,1837,4338, // 3878
150 1820,2817,3635,2747,2283,2002,4339,2936,2748, 144,3321, 882,4340,3892,2749,3423, // 3894
151 4341,2901,7417,4095,1726, 320,7418,3893,3026, 788,2978,7419,2818,1773,1327,2859, // 3910
152 3894,2819,7420,1306,4342,2003,1700,3752,3521,2359,2650, 787,2022, 506, 824,3636, // 3926
153 534, 323,4343,1044,3322,2023,1900, 946,3424,7421,1778,1500,1678,7422,1881,4344, // 3942
154 165, 243,4345,3637,2521, 123, 683,4096, 764,4346, 36,3895,1792, 589,2902, 816, // 3958
155 626,1667,3027,2233,1639,1555,1622,3753,3896,7423,3897,2860,1370,1228,1932, 891, // 3974
156 2083,2903, 304,4097,7424, 292,2979,2711,3522, 691,2100,4098,1115,4347, 118, 662, // 3990
157 7425, 611,1156, 854,2381,1316,2861, 2, 386, 515,2904,7426,7427,3253, 868,2234, // 4006
158 1486, 855,2651, 785,2212,3028,7428,1040,3185,3523,7429,3121, 448,7430,1525,7431, // 4022
159 2164,4348,7432,3754,7433,4099,2820,3524,3122, 503, 818,3898,3123,1568, 814, 676, // 4038
160 1444, 306,1749,7434,3755,1416,1030, 197,1428, 805,2821,1501,4349,7435,7436,7437, // 4054
161 1993,7438,4350,7439,7440,2195, 13,2779,3638,2980,3124,1229,1916,7441,3756,2131, // 4070
162 7442,4100,4351,2399,3525,7443,2213,1511,1727,1120,7444,7445, 646,3757,2443, 307, // 4086
163 7446,7447,1595,3186,7448,7449,7450,3639,1113,1356,3899,1465,2522,2523,7451, 519, // 4102
164 7452, 128,2132, 92,2284,1979,7453,3900,1512, 342,3125,2196,7454,2780,2214,1980, // 4118
165 3323,7455, 290,1656,1317, 789, 827,2360,7456,3758,4352, 562, 581,3901,7457, 401, // 4134
166 4353,2248, 94,4354,1399,2781,7458,1463,2024,4355,3187,1943,7459, 828,1105,4101, // 4150
167 1262,1394,7460,4102, 605,4356,7461,1783,2862,7462,2822, 819,2101, 578,2197,2937, // 4166
168 7463,1502, 436,3254,4103,3255,2823,3902,2905,3425,3426,7464,2712,2315,7465,7466, // 4182
169 2332,2067, 23,4357, 193, 826,3759,2102, 699,1630,4104,3075, 390,1793,1064,3526, // 4198
170 7467,1579,3076,3077,1400,7468,4105,1838,1640,2863,7469,4358,4359, 137,4106, 598, // 4214
171 3078,1966, 780, 104, 974,2938,7470, 278, 899, 253, 402, 572, 504, 493,1339,7471, // 4230
172 3903,1275,4360,2574,2550,7472,3640,3029,3079,2249, 565,1334,2713, 863, 41,7473, // 4246
173 7474,4361,7475,1657,2333, 19, 463,2750,4107, 606,7476,2981,3256,1087,2084,1323, // 4262
174 2652,2982,7477,1631,1623,1750,4108,2682,7478,2864, 791,2714,2653,2334, 232,2416, // 4278
175 7479,2983,1498,7480,2654,2620, 755,1366,3641,3257,3126,2025,1609, 119,1917,3427, // 4294
176 862,1026,4109,7481,3904,3760,4362,3905,4363,2260,1951,2470,7482,1125, 817,4110, // 4310
177 4111,3906,1513,1766,2040,1487,4112,3030,3258,2824,3761,3127,7483,7484,1507,7485, // 4326
178 2683, 733, 40,1632,1106,2865, 345,4113, 841,2524, 230,4364,2984,1846,3259,3428, // 4342
179 7486,1263, 986,3429,7487, 735, 879, 254,1137, 857, 622,1300,1180,1388,1562,3907, // 4358
180 3908,2939, 967,2751,2655,1349, 592,2133,1692,3324,2985,1994,4114,1679,3909,1901, // 4374
181 2185,7488, 739,3642,2715,1296,1290,7489,4115,2198,2199,1921,1563,2595,2551,1870, // 4390
182 2752,2986,7490, 435,7491, 343,1108, 596, 17,1751,4365,2235,3430,3643,7492,4366, // 4406
183 294,3527,2940,1693, 477, 979, 281,2041,3528, 643,2042,3644,2621,2782,2261,1031, // 4422
184 2335,2134,2298,3529,4367, 367,1249,2552,7493,3530,7494,4368,1283,3325,2004, 240, // 4438
185 1762,3326,4369,4370, 836,1069,3128, 474,7495,2148,2525, 268,3531,7496,3188,1521, // 4454
186 1284,7497,1658,1546,4116,7498,3532,3533,7499,4117,3327,2684,1685,4118, 961,1673, // 4470
187 2622, 190,2005,2200,3762,4371,4372,7500, 570,2497,3645,1490,7501,4373,2623,3260, // 4486
188 1956,4374, 584,1514, 396,1045,1944,7502,4375,1967,2444,7503,7504,4376,3910, 619, // 4502
189 7505,3129,3261, 215,2006,2783,2553,3189,4377,3190,4378, 763,4119,3763,4379,7506, // 4518
190 7507,1957,1767,2941,3328,3646,1174, 452,1477,4380,3329,3130,7508,2825,1253,2382, // 4534
191 2186,1091,2285,4120, 492,7509, 638,1169,1824,2135,1752,3911, 648, 926,1021,1324, // 4550
192 4381, 520,4382, 997, 847,1007, 892,4383,3764,2262,1871,3647,7510,2400,1784,4384, // 4566
193 1952,2942,3080,3191,1728,4121,2043,3648,4385,2007,1701,3131,1551, 30,2263,4122, // 4582
194 7511,2026,4386,3534,7512, 501,7513,4123, 594,3431,2165,1821,3535,3432,3536,3192, // 4598
195 829,2826,4124,7514,1680,3132,1225,4125,7515,3262,4387,4126,3133,2336,7516,4388, // 4614
196 4127,7517,3912,3913,7518,1847,2383,2596,3330,7519,4389, 374,3914, 652,4128,4129, // 4630
197 375,1140, 798,7520,7521,7522,2361,4390,2264, 546,1659, 138,3031,2445,4391,7523, // 4646
198 2250, 612,1848, 910, 796,3765,1740,1371, 825,3766,3767,7524,2906,2554,7525, 692, // 4662
199 444,3032,2624, 801,4392,4130,7526,1491, 244,1053,3033,4131,4132, 340,7527,3915, // 4678
200 1041,2987, 293,1168, 87,1357,7528,1539, 959,7529,2236, 721, 694,4133,3768, 219, // 4694
201 1478, 644,1417,3331,2656,1413,1401,1335,1389,3916,7530,7531,2988,2362,3134,1825, // 4710
202 730,1515, 184,2827, 66,4393,7532,1660,2943, 246,3332, 378,1457, 226,3433, 975, // 4726
203 3917,2944,1264,3537, 674, 696,7533, 163,7534,1141,2417,2166, 713,3538,3333,4394, // 4742
204 3918,7535,7536,1186, 15,7537,1079,1070,7538,1522,3193,3539, 276,1050,2716, 758, // 4758
205 1126, 653,2945,3263,7539,2337, 889,3540,3919,3081,2989, 903,1250,4395,3920,3434, // 4774
206 3541,1342,1681,1718, 766,3264, 286, 89,2946,3649,7540,1713,7541,2597,3334,2990, // 4790
207 7542,2947,2215,3194,2866,7543,4396,2498,2526, 181, 387,1075,3921, 731,2187,3335, // 4806
208 7544,3265, 310, 313,3435,2299, 770,4134, 54,3034, 189,4397,3082,3769,3922,7545, // 4822
209 1230,1617,1849, 355,3542,4135,4398,3336, 111,4136,3650,1350,3135,3436,3035,4137, // 4838
210 2149,3266,3543,7546,2784,3923,3924,2991, 722,2008,7547,1071, 247,1207,2338,2471, // 4854
211 1378,4399,2009, 864,1437,1214,4400, 373,3770,1142,2216, 667,4401, 442,2753,2555, // 4870
212 3771,3925,1968,4138,3267,1839, 837, 170,1107, 934,1336,1882,7548,7549,2118,4139, // 4886
213 2828, 743,1569,7550,4402,4140, 582,2384,1418,3437,7551,1802,7552, 357,1395,1729, // 4902
214 3651,3268,2418,1564,2237,7553,3083,3772,1633,4403,1114,2085,4141,1532,7554, 482, // 4918
215 2446,4404,7555,7556,1492, 833,1466,7557,2717,3544,1641,2829,7558,1526,1272,3652, // 4934
216 4142,1686,1794, 416,2556,1902,1953,1803,7559,3773,2785,3774,1159,2316,7560,2867, // 4950
217 4405,1610,1584,3036,2419,2754, 443,3269,1163,3136,7561,7562,3926,7563,4143,2499, // 4966
218 3037,4406,3927,3137,2103,1647,3545,2010,1872,4144,7564,4145, 431,3438,7565, 250, // 4982
219 97, 81,4146,7566,1648,1850,1558, 160, 848,7567, 866, 740,1694,7568,2201,2830, // 4998
220 3195,4147,4407,3653,1687, 950,2472, 426, 469,3196,3654,3655,3928,7569,7570,1188, // 5014
221 424,1995, 861,3546,4148,3775,2202,2685, 168,1235,3547,4149,7571,2086,1674,4408, // 5030
222 3337,3270, 220,2557,1009,7572,3776, 670,2992, 332,1208, 717,7573,7574,3548,2447, // 5046
223 3929,3338,7575, 513,7576,1209,2868,3339,3138,4409,1080,7577,7578,7579,7580,2527, // 5062
224 3656,3549, 815,1587,3930,3931,7581,3550,3439,3777,1254,4410,1328,3038,1390,3932, // 5078
225 1741,3933,3778,3934,7582, 236,3779,2448,3271,7583,7584,3657,3780,1273,3781,4411, // 5094
226 7585, 308,7586,4412, 245,4413,1851,2473,1307,2575, 430, 715,2136,2449,7587, 270, // 5110
227 199,2869,3935,7588,3551,2718,1753, 761,1754, 725,1661,1840,4414,3440,3658,7589, // 5126
228 7590, 587, 14,3272, 227,2598, 326, 480,2265, 943,2755,3552, 291, 650,1883,7591, // 5142
229 1702,1226, 102,1547, 62,3441, 904,4415,3442,1164,4150,7592,7593,1224,1548,2756, // 5158
230 391, 498,1493,7594,1386,1419,7595,2055,1177,4416, 813, 880,1081,2363, 566,1145, // 5174
231 4417,2286,1001,1035,2558,2599,2238, 394,1286,7596,7597,2068,7598, 86,1494,1730, // 5190
232 3936, 491,1588, 745, 897,2948, 843,3340,3937,2757,2870,3273,1768, 998,2217,2069, // 5206
233 397,1826,1195,1969,3659,2993,3341, 284,7599,3782,2500,2137,2119,1903,7600,3938, // 5222
234 2150,3939,4151,1036,3443,1904, 114,2559,4152, 209,1527,7601,7602,2949,2831,2625, // 5238
235 2385,2719,3139, 812,2560,7603,3274,7604,1559, 737,1884,3660,1210, 885, 28,2686, // 5254
236 3553,3783,7605,4153,1004,1779,4418,7606, 346,1981,2218,2687,4419,3784,1742, 797, // 5270
237 1642,3940,1933,1072,1384,2151, 896,3941,3275,3661,3197,2871,3554,7607,2561,1958, // 5286
238 4420,2450,1785,7608,7609,7610,3942,4154,1005,1308,3662,4155,2720,4421,4422,1528, // 5302
239 2600, 161,1178,4156,1982, 987,4423,1101,4157, 631,3943,1157,3198,2420,1343,1241, // 5318
240 1016,2239,2562, 372, 877,2339,2501,1160, 555,1934, 911,3944,7611, 466,1170, 169, // 5334
241 1051,2907,2688,3663,2474,2994,1182,2011,2563,1251,2626,7612, 992,2340,3444,1540, // 5350
242 2721,1201,2070,2401,1996,2475,7613,4424, 528,1922,2188,1503,1873,1570,2364,3342, // 5366
243 3276,7614, 557,1073,7615,1827,3445,2087,2266,3140,3039,3084, 767,3085,2786,4425, // 5382
244 1006,4158,4426,2341,1267,2176,3664,3199, 778,3945,3200,2722,1597,2657,7616,4427, // 5398
245 7617,3446,7618,7619,7620,3277,2689,1433,3278, 131, 95,1504,3946, 723,4159,3141, // 5414
246 1841,3555,2758,2189,3947,2027,2104,3665,7621,2995,3948,1218,7622,3343,3201,3949, // 5430
247 4160,2576, 248,1634,3785, 912,7623,2832,3666,3040,3786, 654, 53,7624,2996,7625, // 5446
248 1688,4428, 777,3447,1032,3950,1425,7626, 191, 820,2120,2833, 971,4429, 931,3202, // 5462
249 135, 664, 783,3787,1997, 772,2908,1935,3951,3788,4430,2909,3203, 282,2723, 640, // 5478
250 1372,3448,1127, 922, 325,3344,7627,7628, 711,2044,7629,7630,3952,2219,2787,1936, // 5494
251 3953,3345,2220,2251,3789,2300,7631,4431,3790,1258,3279,3954,3204,2138,2950,3955, // 5510
252 3956,7632,2221, 258,3205,4432, 101,1227,7633,3280,1755,7634,1391,3281,7635,2910, // 5526
253 2056, 893,7636,7637,7638,1402,4161,2342,7639,7640,3206,3556,7641,7642, 878,1325, // 5542
254 1780,2788,4433, 259,1385,2577, 744,1183,2267,4434,7643,3957,2502,7644, 684,1024, // 5558
255 4162,7645, 472,3557,3449,1165,3282,3958,3959, 322,2152, 881, 455,1695,1152,1340, // 5574
256 660, 554,2153,4435,1058,4436,4163, 830,1065,3346,3960,4437,1923,7646,1703,1918, // 5590
257 7647, 932,2268, 122,7648,4438, 947, 677,7649,3791,2627, 297,1905,1924,2269,4439, // 5606
258 2317,3283,7650,7651,4164,7652,4165, 84,4166, 112, 989,7653, 547,1059,3961, 701, // 5622
259 3558,1019,7654,4167,7655,3450, 942, 639, 457,2301,2451, 993,2951, 407, 851, 494, // 5638
260 4440,3347, 927,7656,1237,7657,2421,3348, 573,4168, 680, 921,2911,1279,1874, 285, // 5654
261 790,1448,1983, 719,2167,7658,7659,4441,3962,3963,1649,7660,1541, 563,7661,1077, // 5670
262 7662,3349,3041,3451, 511,2997,3964,3965,3667,3966,1268,2564,3350,3207,4442,4443, // 5686
263 7663, 535,1048,1276,1189,2912,2028,3142,1438,1373,2834,2952,1134,2012,7664,4169, // 5702
264 1238,2578,3086,1259,7665, 700,7666,2953,3143,3668,4170,7667,4171,1146,1875,1906, // 5718
265 4444,2601,3967, 781,2422, 132,1589, 203, 147, 273,2789,2402, 898,1786,2154,3968, // 5734
266 3969,7668,3792,2790,7669,7670,4445,4446,7671,3208,7672,1635,3793, 965,7673,1804, // 5750
267 2690,1516,3559,1121,1082,1329,3284,3970,1449,3794, 65,1128,2835,2913,2759,1590, // 5766
268 3795,7674,7675, 12,2658, 45, 976,2579,3144,4447, 517,2528,1013,1037,3209,7676, // 5782
269 3796,2836,7677,3797,7678,3452,7679,2602, 614,1998,2318,3798,3087,2724,2628,7680, // 5798
270 2580,4172, 599,1269,7681,1810,3669,7682,2691,3088, 759,1060, 489,1805,3351,3285, // 5814
271 1358,7683,7684,2386,1387,1215,2629,2252, 490,7685,7686,4173,1759,2387,2343,7687, // 5830
272 4448,3799,1907,3971,2630,1806,3210,4449,3453,3286,2760,2344, 874,7688,7689,3454, // 5846
273 3670,1858, 91,2914,3671,3042,3800,4450,7690,3145,3972,2659,7691,3455,1202,1403, // 5862
274 3801,2954,2529,1517,2503,4451,3456,2504,7692,4452,7693,2692,1885,1495,1731,3973, // 5878
275 2365,4453,7694,2029,7695,7696,3974,2693,1216, 237,2581,4174,2319,3975,3802,4454, // 5894
276 4455,2694,3560,3457, 445,4456,7697,7698,7699,7700,2761, 61,3976,3672,1822,3977, // 5910
277 7701, 687,2045, 935, 925, 405,2660, 703,1096,1859,2725,4457,3978,1876,1367,2695, // 5926
278 3352, 918,2105,1781,2476, 334,3287,1611,1093,4458, 564,3146,3458,3673,3353, 945, // 5942
279 2631,2057,4459,7702,1925, 872,4175,7703,3459,2696,3089, 349,4176,3674,3979,4460, // 5958
280 3803,4177,3675,2155,3980,4461,4462,4178,4463,2403,2046, 782,3981, 400, 251,4179, // 5974
281 1624,7704,7705, 277,3676, 299,1265, 476,1191,3804,2121,4180,4181,1109, 205,7706, // 5990
282 2582,1000,2156,3561,1860,7707,7708,7709,4464,7710,4465,2565, 107,2477,2157,3982, // 6006
283 3460,3147,7711,1533, 541,1301, 158, 753,4182,2872,3562,7712,1696, 370,1088,4183, // 6022
284 4466,3563, 579, 327, 440, 162,2240, 269,1937,1374,3461, 968,3043, 56,1396,3090, // 6038
285 2106,3288,3354,7713,1926,2158,4467,2998,7714,3564,7715,7716,3677,4468,2478,7717, // 6054
286 2791,7718,1650,4469,7719,2603,7720,7721,3983,2661,3355,1149,3356,3984,3805,3985, // 6070
287 7722,1076, 49,7723, 951,3211,3289,3290, 450,2837, 920,7724,1811,2792,2366,4184, // 6086
288 1908,1138,2367,3806,3462,7725,3212,4470,1909,1147,1518,2423,4471,3807,7726,4472, // 6102
289 2388,2604, 260,1795,3213,7727,7728,3808,3291, 708,7729,3565,1704,7730,3566,1351, // 6118
290 1618,3357,2999,1886, 944,4185,3358,4186,3044,3359,4187,7731,3678, 422, 413,1714, // 6134
291 3292, 500,2058,2345,4188,2479,7732,1344,1910, 954,7733,1668,7734,7735,3986,2404, // 6150
292 4189,3567,3809,4190,7736,2302,1318,2505,3091, 133,3092,2873,4473, 629, 31,2838, // 6166
293 2697,3810,4474, 850, 949,4475,3987,2955,1732,2088,4191,1496,1852,7737,3988, 620, // 6182
294 3214, 981,1242,3679,3360,1619,3680,1643,3293,2139,2452,1970,1719,3463,2168,7738, // 6198
295 3215,7739,7740,3361,1828,7741,1277,4476,1565,2047,7742,1636,3568,3093,7743, 869, // 6214
296 2839, 655,3811,3812,3094,3989,3000,3813,1310,3569,4477,7744,7745,7746,1733, 558, // 6230
297 4478,3681, 335,1549,3045,1756,4192,3682,1945,3464,1829,1291,1192, 470,2726,2107, // 6246
298 2793, 913,1054,3990,7747,1027,7748,3046,3991,4479, 982,2662,3362,3148,3465,3216, // 6262
299 3217,1946,2794,7749, 571,4480,7750,1830,7751,3570,2583,1523,2424,7752,2089, 984, // 6278
300 4481,3683,1959,7753,3684, 852, 923,2795,3466,3685, 969,1519, 999,2048,2320,1705, // 6294
301 7754,3095, 615,1662, 151, 597,3992,2405,2321,1049, 275,4482,3686,4193, 568,3687, // 6310
302 3571,2480,4194,3688,7755,2425,2270, 409,3218,7756,1566,2874,3467,1002, 769,2840, // 6326
303 194,2090,3149,3689,2222,3294,4195, 628,1505,7757,7758,1763,2177,3001,3993, 521, // 6342
304 1161,2584,1787,2203,2406,4483,3994,1625,4196,4197, 412, 42,3096, 464,7759,2632, // 6358
305 4484,3363,1760,1571,2875,3468,2530,1219,2204,3814,2633,2140,2368,4485,4486,3295, // 6374
306 1651,3364,3572,7760,7761,3573,2481,3469,7762,3690,7763,7764,2271,2091, 460,7765, // 6390
307 4487,7766,3002, 962, 588,3574, 289,3219,2634,1116, 52,7767,3047,1796,7768,7769, // 6406
308 7770,1467,7771,1598,1143,3691,4198,1984,1734,1067,4488,1280,3365, 465,4489,1572, // 6422
309 510,7772,1927,2241,1812,1644,3575,7773,4490,3692,7774,7775,2663,1573,1534,7776, // 6438
310 7777,4199, 536,1807,1761,3470,3815,3150,2635,7778,7779,7780,4491,3471,2915,1911, // 6454
311 2796,7781,3296,1122, 377,3220,7782, 360,7783,7784,4200,1529, 551,7785,2059,3693, // 6470
312 1769,2426,7786,2916,4201,3297,3097,2322,2108,2030,4492,1404, 136,1468,1479, 672, // 6486
313 1171,3221,2303, 271,3151,7787,2762,7788,2049, 678,2727, 865,1947,4493,7789,2013, // 6502
314 3995,2956,7790,2728,2223,1397,3048,3694,4494,4495,1735,2917,3366,3576,7791,3816, // 6518
315 509,2841,2453,2876,3817,7792,7793,3152,3153,4496,4202,2531,4497,2304,1166,1010, // 6534
316 552, 681,1887,7794,7795,2957,2958,3996,1287,1596,1861,3154, 358, 453, 736, 175, // 6550
317 478,1117, 905,1167,1097,7796,1853,1530,7797,1706,7798,2178,3472,2287,3695,3473, // 6566
318 3577,4203,2092,4204,7799,3367,1193,2482,4205,1458,2190,2205,1862,1888,1421,3298, // 6582
319 2918,3049,2179,3474, 595,2122,7800,3997,7801,7802,4206,1707,2636, 223,3696,1359, // 6598
320 751,3098, 183,3475,7803,2797,3003, 419,2369, 633, 704,3818,2389, 241,7804,7805, // 6614
321 7806, 838,3004,3697,2272,2763,2454,3819,1938,2050,3998,1309,3099,2242,1181,7807, // 6630
322 1136,2206,3820,2370,1446,4207,2305,4498,7808,7809,4208,1055,2605, 484,3698,7810, // 6646
323 3999, 625,4209,2273,3368,1499,4210,4000,7811,4001,4211,3222,2274,2275,3476,7812, // 6662
324 7813,2764, 808,2606,3699,3369,4002,4212,3100,2532, 526,3370,3821,4213, 955,7814, // 6678
325 1620,4214,2637,2427,7815,1429,3700,1669,1831, 994, 928,7816,3578,1260,7817,7818, // 6694
326 7819,1948,2288, 741,2919,1626,4215,2729,2455, 867,1184, 362,3371,1392,7820,7821, // 6710
327 4003,4216,1770,1736,3223,2920,4499,4500,1928,2698,1459,1158,7822,3050,3372,2877, // 6726
328 1292,1929,2506,2842,3701,1985,1187,2071,2014,2607,4217,7823,2566,2507,2169,3702, // 6742
329 2483,3299,7824,3703,4501,7825,7826, 666,1003,3005,1022,3579,4218,7827,4502,1813, // 6758
330 2253, 574,3822,1603, 295,1535, 705,3823,4219, 283, 858, 417,7828,7829,3224,4503, // 6774
331 4504,3051,1220,1889,1046,2276,2456,4004,1393,1599, 689,2567, 388,4220,7830,2484, // 6790
332 802,7831,2798,3824,2060,1405,2254,7832,4505,3825,2109,1052,1345,3225,1585,7833, // 6806
333 809,7834,7835,7836, 575,2730,3477, 956,1552,1469,1144,2323,7837,2324,1560,2457, // 6822
334 3580,3226,4005, 616,2207,3155,2180,2289,7838,1832,7839,3478,4506,7840,1319,3704, // 6838
335 3705,1211,3581,1023,3227,1293,2799,7841,7842,7843,3826, 607,2306,3827, 762,2878, // 6854
336 1439,4221,1360,7844,1485,3052,7845,4507,1038,4222,1450,2061,2638,4223,1379,4508, // 6870
337 2585,7846,7847,4224,1352,1414,2325,2921,1172,7848,7849,3828,3829,7850,1797,1451, // 6886
338 7851,7852,7853,7854,2922,4006,4007,2485,2346, 411,4008,4009,3582,3300,3101,4509, // 6902
339 1561,2664,1452,4010,1375,7855,7856, 47,2959, 316,7857,1406,1591,2923,3156,7858, // 6918
340 1025,2141,3102,3157, 354,2731, 884,2224,4225,2407, 508,3706, 726,3583, 996,2428, // 6934
341 3584, 729,7859, 392,2191,1453,4011,4510,3707,7860,7861,2458,3585,2608,1675,2800, // 6950
342 919,2347,2960,2348,1270,4511,4012, 73,7862,7863, 647,7864,3228,2843,2255,1550, // 6966
343 1346,3006,7865,1332, 883,3479,7866,7867,7868,7869,3301,2765,7870,1212, 831,1347, // 6982
344 4226,4512,2326,3830,1863,3053, 720,3831,4513,4514,3832,7871,4227,7872,7873,4515, // 6998
345 7874,7875,1798,4516,3708,2609,4517,3586,1645,2371,7876,7877,2924, 669,2208,2665, // 7014
346 2429,7878,2879,7879,7880,1028,3229,7881,4228,2408,7882,2256,1353,7883,7884,4518, // 7030
347 3158, 518,7885,4013,7886,4229,1960,7887,2142,4230,7888,7889,3007,2349,2350,3833, // 7046
348 516,1833,1454,4014,2699,4231,4519,2225,2610,1971,1129,3587,7890,2766,7891,2961, // 7062
349 1422, 577,1470,3008,1524,3373,7892,7893, 432,4232,3054,3480,7894,2586,1455,2508, // 7078
350 2226,1972,1175,7895,1020,2732,4015,3481,4520,7896,2733,7897,1743,1361,3055,3482, // 7094
351 2639,4016,4233,4521,2290, 895, 924,4234,2170, 331,2243,3056, 166,1627,3057,1098, // 7110
352 7898,1232,2880,2227,3374,4522, 657, 403,1196,2372, 542,3709,3375,1600,4235,3483, // 7126
353 7899,4523,2767,3230, 576, 530,1362,7900,4524,2533,2666,3710,4017,7901, 842,3834, // 7142
354 7902,2801,2031,1014,4018, 213,2700,3376, 665, 621,4236,7903,3711,2925,2430,7904, // 7158
355 2431,3302,3588,3377,7905,4237,2534,4238,4525,3589,1682,4239,3484,1380,7906, 724, // 7174
356 2277, 600,1670,7907,1337,1233,4526,3103,2244,7908,1621,4527,7909, 651,4240,7910, // 7190
357 1612,4241,2611,7911,2844,7912,2734,2307,3058,7913, 716,2459,3059, 174,1255,2701, // 7206
358 4019,3590, 548,1320,1398, 728,4020,1574,7914,1890,1197,3060,4021,7915,3061,3062, // 7222
359 3712,3591,3713, 747,7916, 635,4242,4528,7917,7918,7919,4243,7920,7921,4529,7922, // 7238
360 3378,4530,2432, 451,7923,3714,2535,2072,4244,2735,4245,4022,7924,1764,4531,7925, // 7254
361 4246, 350,7926,2278,2390,2486,7927,4247,4023,2245,1434,4024, 488,4532, 458,4248, // 7270
362 4025,3715, 771,1330,2391,3835,2568,3159,2159,2409,1553,2667,3160,4249,7928,2487, // 7286
363 2881,2612,1720,2702,4250,3379,4533,7929,2536,4251,7930,3231,4252,2768,7931,2015, // 7302
364 2736,7932,1155,1017,3716,3836,7933,3303,2308, 201,1864,4253,1430,7934,4026,7935, // 7318
365 7936,7937,7938,7939,4254,1604,7940, 414,1865, 371,2587,4534,4535,3485,2016,3104, // 7334
366 4536,1708, 960,4255, 887, 389,2171,1536,1663,1721,7941,2228,4027,2351,2926,1580, // 7350
367 7942,7943,7944,1744,7945,2537,4537,4538,7946,4539,7947,2073,7948,7949,3592,3380, // 7366
368 2882,4256,7950,4257,2640,3381,2802, 673,2703,2460, 709,3486,4028,3593,4258,7951, // 7382
369 1148, 502, 634,7952,7953,1204,4540,3594,1575,4541,2613,3717,7954,3718,3105, 948, // 7398
370 3232, 121,1745,3837,1110,7955,4259,3063,2509,3009,4029,3719,1151,1771,3838,1488, // 7414
371 4030,1986,7956,2433,3487,7957,7958,2093,7959,4260,3839,1213,1407,2803, 531,2737, // 7430
372 2538,3233,1011,1537,7960,2769,4261,3106,1061,7961,3720,3721,1866,2883,7962,2017, // 7446
373 120,4262,4263,2062,3595,3234,2309,3840,2668,3382,1954,4542,7963,7964,3488,1047, // 7462
374 2704,1266,7965,1368,4543,2845, 649,3383,3841,2539,2738,1102,2846,2669,7966,7967, // 7478
375 1999,7968,1111,3596,2962,7969,2488,3842,3597,2804,1854,3384,3722,7970,7971,3385, // 7494
376 2410,2884,3304,3235,3598,7972,2569,7973,3599,2805,4031,1460, 856,7974,3600,7975, // 7510
377 2885,2963,7976,2886,3843,7977,4264, 632,2510, 875,3844,1697,3845,2291,7978,7979, // 7526
378 4544,3010,1239, 580,4545,4265,7980, 914, 936,2074,1190,4032,1039,2123,7981,7982, // 7542
379 7983,3386,1473,7984,1354,4266,3846,7985,2172,3064,4033, 915,3305,4267,4268,3306, // 7558
380 1605,1834,7986,2739, 398,3601,4269,3847,4034, 328,1912,2847,4035,3848,1331,4270, // 7574
381 3011, 937,4271,7987,3602,4036,4037,3387,2160,4546,3388, 524, 742, 538,3065,1012, // 7590
382 7988,7989,3849,2461,7990, 658,1103, 225,3850,7991,7992,4547,7993,4548,7994,3236, // 7606
383 1243,7995,4038, 963,2246,4549,7996,2705,3603,3161,7997,7998,2588,2327,7999,4550, // 7622
384 8000,8001,8002,3489,3307, 957,3389,2540,2032,1930,2927,2462, 870,2018,3604,1746, // 7638
385 2770,2771,2434,2463,8003,3851,8004,3723,3107,3724,3490,3390,3725,8005,1179,3066, // 7654
386 8006,3162,2373,4272,3726,2541,3163,3108,2740,4039,8007,3391,1556,2542,2292, 977, // 7670
387 2887,2033,4040,1205,3392,8008,1765,3393,3164,2124,1271,1689, 714,4551,3491,8009, // 7686
388 2328,3852, 533,4273,3605,2181, 617,8010,2464,3308,3492,2310,8011,8012,3165,8013, // 7702
389 8014,3853,1987, 618, 427,2641,3493,3394,8015,8016,1244,1690,8017,2806,4274,4552, // 7718
390 8018,3494,8019,8020,2279,1576, 473,3606,4275,3395, 972,8021,3607,8022,3067,8023, // 7734
391 8024,4553,4554,8025,3727,4041,4042,8026, 153,4555, 356,8027,1891,2888,4276,2143, // 7750
392 408, 803,2352,8028,3854,8029,4277,1646,2570,2511,4556,4557,3855,8030,3856,4278, // 7766
393 8031,2411,3396, 752,8032,8033,1961,2964,8034, 746,3012,2465,8035,4279,3728, 698, // 7782
394 4558,1892,4280,3608,2543,4559,3609,3857,8036,3166,3397,8037,1823,1302,4043,2706, // 7798
395 3858,1973,4281,8038,4282,3167, 823,1303,1288,1236,2848,3495,4044,3398, 774,3859, // 7814
396 8039,1581,4560,1304,2849,3860,4561,8040,2435,2161,1083,3237,4283,4045,4284, 344, // 7830
397 1173, 288,2311, 454,1683,8041,8042,1461,4562,4046,2589,8043,8044,4563, 985, 894, // 7846
398 8045,3399,3168,8046,1913,2928,3729,1988,8047,2110,1974,8048,4047,8049,2571,1194, // 7862
399 425,8050,4564,3169,1245,3730,4285,8051,8052,2850,8053, 636,4565,1855,3861, 760, // 7878
400 1799,8054,4286,2209,1508,4566,4048,1893,1684,2293,8055,8056,8057,4287,4288,2210, // 7894
401 479,8058,8059, 832,8060,4049,2489,8061,2965,2490,3731, 990,3109, 627,1814,2642, // 7910
402 4289,1582,4290,2125,2111,3496,4567,8062, 799,4291,3170,8063,4568,2112,1737,3013, // 7926
403 1018, 543, 754,4292,3309,1676,4569,4570,4050,8064,1489,8065,3497,8066,2614,2889, // 7942
404 4051,8067,8068,2966,8069,8070,8071,8072,3171,4571,4572,2182,1722,8073,3238,3239, // 7958
405 1842,3610,1715, 481, 365,1975,1856,8074,8075,1962,2491,4573,8076,2126,3611,3240, // 7974
406 433,1894,2063,2075,8077, 602,2741,8078,8079,8080,8081,8082,3014,1628,3400,8083, // 7990
407 3172,4574,4052,2890,4575,2512,8084,2544,2772,8085,8086,8087,3310,4576,2891,8088, // 8006
408 4577,8089,2851,4578,4579,1221,2967,4053,2513,8090,8091,8092,1867,1989,8093,8094, // 8022
409 8095,1895,8096,8097,4580,1896,4054, 318,8098,2094,4055,4293,8099,8100, 485,8101, // 8038
410 938,3862, 553,2670, 116,8102,3863,3612,8103,3498,2671,2773,3401,3311,2807,8104, // 8054
411 3613,2929,4056,1747,2930,2968,8105,8106, 207,8107,8108,2672,4581,2514,8109,3015, // 8070
412 890,3614,3864,8110,1877,3732,3402,8111,2183,2353,3403,1652,8112,8113,8114, 941, // 8086
413 2294, 208,3499,4057,2019, 330,4294,3865,2892,2492,3733,4295,8115,8116,8117,8118, // 8102
414
415 /***************************************************************************************
416 *Everything below is of no interest for detection purpose *
417 ***************************************************************************************
418
419 2515,1613,4582,8119,3312,3866,2516,8120,4058,8121,1637,4059,2466,4583,3867,8122, // 8118
420 2493,3016,3734,8123,8124,2192,8125,8126,2162,8127,8128,8129,8130,8131,8132,8133, // 8134
421 8134,8135,8136,8137,8138,8139,8140,8141,8142,8143,8144,8145,8146,8147,8148,8149, // 8150
422 8150,8151,8152,8153,8154,8155,8156,8157,8158,8159,8160,8161,8162,8163,8164,8165, // 8166
423 8166,8167,8168,8169,8170,8171,8172,8173,8174,8175,8176,8177,8178,8179,8180,8181, // 8182
424 8182,8183,8184,8185,8186,8187,8188,8189,8190,8191,8192,8193,8194,8195,8196,8197, // 8198
425 8198,8199,8200,8201,8202,8203,8204,8205,8206,8207,8208,8209,8210,8211,8212,8213, // 8214
426 8214,8215,8216,8217,8218,8219,8220,8221,8222,8223,8224,8225,8226,8227,8228,8229, // 8230
427 8230,8231,8232,8233,8234,8235,8236,8237,8238,8239,8240,8241,8242,8243,8244,8245, // 8246
428 8246,8247,8248,8249,8250,8251,8252,8253,8254,8255,8256,8257,8258,8259,8260,8261, // 8262
429 8262,8263,8264,8265,8266,8267,8268,8269,8270,8271,8272,8273,8274,8275,8276,8277, // 8278
430 8278,8279,8280,8281,8282,8283,8284,8285,8286,8287,8288,8289,8290,8291,8292,8293, // 8294
431 8294,8295,8296,8297,8298,8299,8300,8301,8302,8303,8304,8305,8306,8307,8308,8309, // 8310
432 8310,8311,8312,8313,8314,8315,8316,8317,8318,8319,8320,8321,8322,8323,8324,8325, // 8326
433 8326,8327,8328,8329,8330,8331,8332,8333,8334,8335,8336,8337,8338,8339,8340,8341, // 8342
434 8342,8343,8344,8345,8346,8347,8348,8349,8350,8351,8352,8353,8354,8355,8356,8357, // 8358
435 8358,8359,8360,8361,8362,8363,8364,8365,8366,8367,8368,8369,8370,8371,8372,8373, // 8374
436 8374,8375,8376,8377,8378,8379,8380,8381,8382,8383,8384,8385,8386,8387,8388,8389, // 8390
437 8390,8391,8392,8393,8394,8395,8396,8397,8398,8399,8400,8401,8402,8403,8404,8405, // 8406
438 8406,8407,8408,8409,8410,8411,8412,8413,8414,8415,8416,8417,8418,8419,8420,8421, // 8422
439 8422,8423,8424,8425,8426,8427,8428,8429,8430,8431,8432,8433,8434,8435,8436,8437, // 8438
440 8438,8439,8440,8441,8442,8443,8444,8445,8446,8447,8448,8449,8450,8451,8452,8453, // 8454
441 8454,8455,8456,8457,8458,8459,8460,8461,8462,8463,8464,8465,8466,8467,8468,8469, // 8470
442 8470,8471,8472,8473,8474,8475,8476,8477,8478,8479,8480,8481,8482,8483,8484,8485, // 8486
443 8486,8487,8488,8489,8490,8491,8492,8493,8494,8495,8496,8497,8498,8499,8500,8501, // 8502
444 8502,8503,8504,8505,8506,8507,8508,8509,8510,8511,8512,8513,8514,8515,8516,8517, // 8518
445 8518,8519,8520,8521,8522,8523,8524,8525,8526,8527,8528,8529,8530,8531,8532,8533, // 8534
446 8534,8535,8536,8537,8538,8539,8540,8541,8542,8543,8544,8545,8546,8547,8548,8549, // 8550
447 8550,8551,8552,8553,8554,8555,8556,8557,8558,8559,8560,8561,8562,8563,8564,8565, // 8566
448 8566,8567,8568,8569,8570,8571,8572,8573,8574,8575,8576,8577,8578,8579,8580,8581, // 8582
449 8582,8583,8584,8585,8586,8587,8588,8589,8590,8591,8592,8593,8594,8595,8596,8597, // 8598
450 8598,8599,8600,8601,8602,8603,8604,8605,8606,8607,8608,8609,8610,8611,8612,8613, // 8614
451 8614,8615,8616,8617,8618,8619,8620,8621,8622,8623,8624,8625,8626,8627,8628,8629, // 8630
452 8630,8631,8632,8633,8634,8635,8636,8637,8638,8639,8640,8641,8642,8643,8644,8645, // 8646
453 8646,8647,8648,8649,8650,8651,8652,8653,8654,8655,8656,8657,8658,8659,8660,8661, // 8662
454 8662,8663,8664,8665,8666,8667,8668,8669,8670,8671,8672,8673,8674,8675,8676,8677, // 8678
455 8678,8679,8680,8681,8682,8683,8684,8685,8686,8687,8688,8689,8690,8691,8692,8693, // 8694
456 8694,8695,8696,8697,8698,8699,8700,8701,8702,8703,8704,8705,8706,8707,8708,8709, // 8710
457 8710,8711,8712,8713,8714,8715,8716,8717,8718,8719,8720,8721,8722,8723,8724,8725, // 8726
458 8726,8727,8728,8729,8730,8731,8732,8733,8734,8735,8736,8737,8738,8739,8740,8741, // 8742
459 ****************************************************************************************/
460 };
461 }
+0
-511
src/org/mozilla/universalchardet/prober/distributionanalysis/GB2312DistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public class GB2312DistributionAnalysis extends CharDistributionAnalysis
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final float GB2312_TYPICAL_DISTRIBUTION_RATIO = 0.9f;
45 public static final int HIGHBYTE_BEGIN = 0xB0;
46 public static final int HIGHBYTE_END = 0xFE;
47 public static final int LOWBYTE_BEGIN = 0xA1;
48 public static final int LOWBYTE_END = 0xFE;
49
50
51 ////////////////////////////////////////////////////////////////
52 // methods
53 ////////////////////////////////////////////////////////////////
54 public GB2312DistributionAnalysis()
55 {
56 super();
57 this.charToFreqOrder = gb2312CharToFreqOrder;
58 this.typicalDistributionRatio = GB2312_TYPICAL_DISTRIBUTION_RATIO;
59 }
60
61 @Override
62 protected int getOrder(byte[] buf, int offset)
63 {
64 int highbyte = buf[offset] & 0xFF;
65 if (highbyte >= HIGHBYTE_BEGIN) {
66 int lowbyte = buf[offset+1] & 0xFF;
67 if (lowbyte >= LOWBYTE_BEGIN) {
68 return (94 * (highbyte - HIGHBYTE_BEGIN) + lowbyte - LOWBYTE_BEGIN);
69 }
70 }
71
72 return -1;
73 }
74
75
76 ////////////////////////////////////////////////////////////////
77 // constants continued
78 ////////////////////////////////////////////////////////////////
79 private static final int[] gb2312CharToFreqOrder = new int[] {
80 1671, 749,1443,2364,3924,3807,2330,3921,1704,3463,2691,1511,1515, 572,3191,2205,
81 2361, 224,2558, 479,1711, 963,3162, 440,4060,1905,2966,2947,3580,2647,3961,3842,
82 2204, 869,4207, 970,2678,5626,2944,2956,1479,4048, 514,3595, 588,1346,2820,3409,
83 249,4088,1746,1873,2047,1774, 581,1813, 358,1174,3590,1014,1561,4844,2245, 670,
84 1636,3112, 889,1286, 953, 556,2327,3060,1290,3141, 613, 185,3477,1367, 850,3820,
85 1715,2428,2642,2303,2732,3041,2562,2648,3566,3946,1349, 388,3098,2091,1360,3585,
86 152,1687,1539, 738,1559, 59,1232,2925,2267,1388,1249,1741,1679,2960, 151,1566,
87 1125,1352,4271, 924,4296, 385,3166,4459, 310,1245,2850, 70,3285,2729,3534,3575,
88 2398,3298,3466,1960,2265, 217,3647, 864,1909,2084,4401,2773,1010,3269,5152, 853,
89 3051,3121,1244,4251,1895, 364,1499,1540,2313,1180,3655,2268, 562, 715,2417,3061,
90 544, 336,3768,2380,1752,4075, 950, 280,2425,4382, 183,2759,3272, 333,4297,2155,
91 1688,2356,1444,1039,4540, 736,1177,3349,2443,2368,2144,2225, 565, 196,1482,3406,
92 927,1335,4147, 692, 878,1311,1653,3911,3622,1378,4200,1840,2969,3149,2126,1816,
93 2534,1546,2393,2760, 737,2494, 13, 447, 245,2747, 38,2765,2129,2589,1079, 606,
94 360, 471,3755,2890, 404, 848, 699,1785,1236, 370,2221,1023,3746,2074,2026,2023,
95 2388,1581,2119, 812,1141,3091,2536,1519, 804,2053, 406,1596,1090, 784, 548,4414,
96 1806,2264,2936,1100, 343,4114,5096, 622,3358, 743,3668,1510,1626,5020,3567,2513,
97 3195,4115,5627,2489,2991, 24,2065,2697,1087,2719, 48,1634, 315, 68, 985,2052,
98 198,2239,1347,1107,1439, 597,2366,2172, 871,3307, 919,2487,2790,1867, 236,2570,
99 1413,3794, 906,3365,3381,1701,1982,1818,1524,2924,1205, 616,2586,2072,2004, 575,
100 253,3099, 32,1365,1182, 197,1714,2454,1201, 554,3388,3224,2748, 756,2587, 250,
101 2567,1507,1517,3529,1922,2761,2337,3416,1961,1677,2452,2238,3153, 615, 911,1506,
102 1474,2495,1265,1906,2749,3756,3280,2161, 898,2714,1759,3450,2243,2444, 563, 26,
103 3286,2266,3769,3344,2707,3677, 611,1402, 531,1028,2871,4548,1375, 261,2948, 835,
104 1190,4134, 353, 840,2684,1900,3082,1435,2109,1207,1674, 329,1872,2781,4055,2686,
105 2104, 608,3318,2423,2957,2768,1108,3739,3512,3271,3985,2203,1771,3520,1418,2054,
106 1681,1153, 225,1627,2929, 162,2050,2511,3687,1954, 124,1859,2431,1684,3032,2894,
107 585,4805,3969,2869,2704,2088,2032,2095,3656,2635,4362,2209, 256, 518,2042,2105,
108 3777,3657, 643,2298,1148,1779, 190, 989,3544, 414, 11,2135,2063,2979,1471, 403,
109 3678, 126, 770,1563, 671,2499,3216,2877, 600,1179, 307,2805,4937,1268,1297,2694,
110 252,4032,1448,1494,1331,1394, 127,2256, 222,1647,1035,1481,3056,1915,1048, 873,
111 3651, 210, 33,1608,2516, 200,1520, 415, 102, 0,3389,1287, 817, 91,3299,2940,
112 836,1814, 549,2197,1396,1669,2987,3582,2297,2848,4528,1070, 687, 20,1819, 121,
113 1552,1364,1461,1968,2617,3540,2824,2083, 177, 948,4938,2291, 110,4549,2066, 648,
114 3359,1755,2110,2114,4642,4845,1693,3937,3308,1257,1869,2123, 208,1804,3159,2992,
115 2531,2549,3361,2418,1350,2347,2800,2568,1291,2036,2680, 72, 842,1990, 212,1233,
116 1154,1586, 75,2027,3410,4900,1823,1337,2710,2676, 728,2810,1522,3026,4995, 157,
117 755,1050,4022, 710, 785,1936,2194,2085,1406,2777,2400, 150,1250,4049,1206, 807,
118 1910, 534, 529,3309,1721,1660, 274, 39,2827, 661,2670,1578, 925,3248,3815,1094,
119 4278,4901,4252, 41,1150,3747,2572,2227,4501,3658,4902,3813,3357,3617,2884,2258,
120 887, 538,4187,3199,1294,2439,3042,2329,2343,2497,1255, 107, 543,1527, 521,3478,
121 3568, 194,5062, 15, 961,3870,1241,1192,2664, 66,5215,3260,2111,1295,1127,2152,
122 3805,4135, 901,1164,1976, 398,1278, 530,1460, 748, 904,1054,1966,1426, 53,2909,
123 509, 523,2279,1534, 536,1019, 239,1685, 460,2353, 673,1065,2401,3600,4298,2272,
124 1272,2363, 284,1753,3679,4064,1695, 81, 815,2677,2757,2731,1386, 859, 500,4221,
125 2190,2566, 757,1006,2519,2068,1166,1455, 337,2654,3203,1863,1682,1914,3025,1252,
126 1409,1366, 847, 714,2834,2038,3209, 964,2970,1901, 885,2553,1078,1756,3049, 301,
127 1572,3326, 688,2130,1996,2429,1805,1648,2930,3421,2750,3652,3088, 262,1158,1254,
128 389,1641,1812, 526,1719, 923,2073,1073,1902, 468, 489,4625,1140, 857,2375,3070,
129 3319,2863, 380, 116,1328,2693,1161,2244, 273,1212,1884,2769,3011,1775,1142, 461,
130 3066,1200,2147,2212, 790, 702,2695,4222,1601,1058, 434,2338,5153,3640, 67,2360,
131 4099,2502, 618,3472,1329, 416,1132, 830,2782,1807,2653,3211,3510,1662, 192,2124,
132 296,3979,1739,1611,3684, 23, 118, 324, 446,1239,1225, 293,2520,3814,3795,2535,
133 3116, 17,1074, 467,2692,2201, 387,2922, 45,1326,3055,1645,3659,2817, 958, 243,
134 1903,2320,1339,2825,1784,3289, 356, 576, 865,2315,2381,3377,3916,1088,3122,1713,
135 1655, 935, 628,4689,1034,1327, 441, 800, 720, 894,1979,2183,1528,5289,2702,1071,
136 4046,3572,2399,1571,3281, 79, 761,1103, 327, 134, 758,1899,1371,1615, 879, 442,
137 215,2605,2579, 173,2048,2485,1057,2975,3317,1097,2253,3801,4263,1403,1650,2946,
138 814,4968,3487,1548,2644,1567,1285, 2, 295,2636, 97, 946,3576, 832, 141,4257,
139 3273, 760,3821,3521,3156,2607, 949,1024,1733,1516,1803,1920,2125,2283,2665,3180,
140 1501,2064,3560,2171,1592, 803,3518,1416, 732,3897,4258,1363,1362,2458, 119,1427,
141 602,1525,2608,1605,1639,3175, 694,3064, 10, 465, 76,2000,4846,4208, 444,3781,
142 1619,3353,2206,1273,3796, 740,2483, 320,1723,2377,3660,2619,1359,1137,1762,1724,
143 2345,2842,1850,1862, 912, 821,1866, 612,2625,1735,2573,3369,1093, 844, 89, 937,
144 930,1424,3564,2413,2972,1004,3046,3019,2011, 711,3171,1452,4178, 428, 801,1943,
145 432, 445,2811, 206,4136,1472, 730, 349, 73, 397,2802,2547, 998,1637,1167, 789,
146 396,3217, 154,1218, 716,1120,1780,2819,4826,1931,3334,3762,2139,1215,2627, 552,
147 3664,3628,3232,1405,2383,3111,1356,2652,3577,3320,3101,1703, 640,1045,1370,1246,
148 4996, 371,1575,2436,1621,2210, 984,4033,1734,2638, 16,4529, 663,2755,3255,1451,
149 3917,2257,1253,1955,2234,1263,2951, 214,1229, 617, 485, 359,1831,1969, 473,2310,
150 750,2058, 165, 80,2864,2419, 361,4344,2416,2479,1134, 796,3726,1266,2943, 860,
151 2715, 938, 390,2734,1313,1384, 248, 202, 877,1064,2854, 522,3907, 279,1602, 297,
152 2357, 395,3740, 137,2075, 944,4089,2584,1267,3802, 62,1533,2285, 178, 176, 780,
153 2440, 201,3707, 590, 478,1560,4354,2117,1075, 30, 74,4643,4004,1635,1441,2745,
154 776,2596, 238,1077,1692,1912,2844, 605, 499,1742,3947, 241,3053, 980,1749, 936,
155 2640,4511,2582, 515,1543,2162,5322,2892,2993, 890,2148,1924, 665,1827,3581,1032,
156 968,3163, 339,1044,1896, 270, 583,1791,1720,4367,1194,3488,3669, 43,2523,1657,
157 163,2167, 290,1209,1622,3378, 550, 634,2508,2510, 695,2634,2384,2512,1476,1414,
158 220,1469,2341,2138,2852,3183,2900,4939,2865,3502,1211,3680, 854,3227,1299,2976,
159 3172, 186,2998,1459, 443,1067,3251,1495, 321,1932,3054, 909, 753,1410,1828, 436,
160 2441,1119,1587,3164,2186,1258, 227, 231,1425,1890,3200,3942, 247, 959, 725,5254,
161 2741, 577,2158,2079, 929, 120, 174, 838,2813, 591,1115, 417,2024, 40,3240,1536,
162 1037, 291,4151,2354, 632,1298,2406,2500,3535,1825,1846,3451, 205,1171, 345,4238,
163 18,1163, 811, 685,2208,1217, 425,1312,1508,1175,4308,2552,1033, 587,1381,3059,
164 2984,3482, 340,1316,4023,3972, 792,3176, 519, 777,4690, 918, 933,4130,2981,3741,
165 90,3360,2911,2200,5184,4550, 609,3079,2030, 272,3379,2736, 363,3881,1130,1447,
166 286, 779, 357,1169,3350,3137,1630,1220,2687,2391, 747,1277,3688,2618,2682,2601,
167 1156,3196,5290,4034,3102,1689,3596,3128, 874, 219,2783, 798, 508,1843,2461, 269,
168 1658,1776,1392,1913,2983,3287,2866,2159,2372, 829,4076, 46,4253,2873,1889,1894,
169 915,1834,1631,2181,2318, 298, 664,2818,3555,2735, 954,3228,3117, 527,3511,2173,
170 681,2712,3033,2247,2346,3467,1652, 155,2164,3382, 113,1994, 450, 899, 494, 994,
171 1237,2958,1875,2336,1926,3727, 545,1577,1550, 633,3473, 204,1305,3072,2410,1956,
172 2471, 707,2134, 841,2195,2196,2663,3843,1026,4940, 990,3252,4997, 368,1092, 437,
173 3212,3258,1933,1829, 675,2977,2893, 412, 943,3723,4644,3294,3283,2230,2373,5154,
174 2389,2241,2661,2323,1404,2524, 593, 787, 677,3008,1275,2059, 438,2709,2609,2240,
175 2269,2246,1446, 36,1568,1373,3892,1574,2301,1456,3962, 693,2276,5216,2035,1143,
176 2720,1919,1797,1811,2763,4137,2597,1830,1699,1488,1198,2090, 424,1694, 312,3634,
177 3390,4179,3335,2252,1214, 561,1059,3243,2295,2561, 975,5155,2321,2751,3772, 472,
178 1537,3282,3398,1047,2077,2348,2878,1323,3340,3076, 690,2906, 51, 369, 170,3541,
179 1060,2187,2688,3670,2541,1083,1683, 928,3918, 459, 109,4427, 599,3744,4286, 143,
180 2101,2730,2490, 82,1588,3036,2121, 281,1860, 477,4035,1238,2812,3020,2716,3312,
181 1530,2188,2055,1317, 843, 636,1808,1173,3495, 649, 181,1002, 147,3641,1159,2414,
182 3750,2289,2795, 813,3123,2610,1136,4368, 5,3391,4541,2174, 420, 429,1728, 754,
183 1228,2115,2219, 347,2223,2733, 735,1518,3003,2355,3134,1764,3948,3329,1888,2424,
184 1001,1234,1972,3321,3363,1672,1021,1450,1584, 226, 765, 655,2526,3404,3244,2302,
185 3665, 731, 594,2184, 319,1576, 621, 658,2656,4299,2099,3864,1279,2071,2598,2739,
186 795,3086,3699,3908,1707,2352,2402,1382,3136,2475,1465,4847,3496,3865,1085,3004,
187 2591,1084, 213,2287,1963,3565,2250, 822, 793,4574,3187,1772,1789,3050, 595,1484,
188 1959,2770,1080,2650, 456, 422,2996, 940,3322,4328,4345,3092,2742, 965,2784, 739,
189 4124, 952,1358,2498,2949,2565, 332,2698,2378, 660,2260,2473,4194,3856,2919, 535,
190 1260,2651,1208,1428,1300,1949,1303,2942, 433,2455,2450,1251,1946, 614,1269, 641,
191 1306,1810,2737,3078,2912, 564,2365,1419,1415,1497,4460,2367,2185,1379,3005,1307,
192 3218,2175,1897,3063, 682,1157,4040,4005,1712,1160,1941,1399, 394, 402,2952,1573,
193 1151,2986,2404, 862, 299,2033,1489,3006, 346, 171,2886,3401,1726,2932, 168,2533,
194 47,2507,1030,3735,1145,3370,1395,1318,1579,3609,4560,2857,4116,1457,2529,1965,
195 504,1036,2690,2988,2405, 745,5871, 849,2397,2056,3081, 863,2359,3857,2096, 99,
196 1397,1769,2300,4428,1643,3455,1978,1757,3718,1440, 35,4879,3742,1296,4228,2280,
197 160,5063,1599,2013, 166, 520,3479,1646,3345,3012, 490,1937,1545,1264,2182,2505,
198 1096,1188,1369,1436,2421,1667,2792,2460,1270,2122, 727,3167,2143, 806,1706,1012,
199 1800,3037, 960,2218,1882, 805, 139,2456,1139,1521, 851,1052,3093,3089, 342,2039,
200 744,5097,1468,1502,1585,2087, 223, 939, 326,2140,2577, 892,2481,1623,4077, 982,
201 3708, 135,2131, 87,2503,3114,2326,1106, 876,1616, 547,2997,2831,2093,3441,4530,
202 4314, 9,3256,4229,4148, 659,1462,1986,1710,2046,2913,2231,4090,4880,5255,3392,
203 3274,1368,3689,4645,1477, 705,3384,3635,1068,1529,2941,1458,3782,1509, 100,1656,
204 2548, 718,2339, 408,1590,2780,3548,1838,4117,3719,1345,3530, 717,3442,2778,3220,
205 2898,1892,4590,3614,3371,2043,1998,1224,3483, 891, 635, 584,2559,3355, 733,1766,
206 1729,1172,3789,1891,2307, 781,2982,2271,1957,1580,5773,2633,2005,4195,3097,1535,
207 3213,1189,1934,5693,3262, 586,3118,1324,1598, 517,1564,2217,1868,1893,4445,3728,
208 2703,3139,1526,1787,1992,3882,2875,1549,1199,1056,2224,1904,2711,5098,4287, 338,
209 1993,3129,3489,2689,1809,2815,1997, 957,1855,3898,2550,3275,3057,1105,1319, 627,
210 1505,1911,1883,3526, 698,3629,3456,1833,1431, 746, 77,1261,2017,2296,1977,1885,
211 125,1334,1600, 525,1798,1109,2222,1470,1945, 559,2236,1186,3443,2476,1929,1411,
212 2411,3135,1777,3372,2621,1841,1613,3229, 668,1430,1839,2643,2916, 195,1989,2671,
213 2358,1387, 629,3205,2293,5256,4439, 123,1310, 888,1879,4300,3021,3605,1003,1162,
214 3192,2910,2010, 140,2395,2859, 55,1082,2012,2901, 662, 419,2081,1438, 680,2774,
215 4654,3912,1620,1731,1625,5035,4065,2328, 512,1344, 802,5443,2163,2311,2537, 524,
216 3399, 98,1155,2103,1918,2606,3925,2816,1393,2465,1504,3773,2177,3963,1478,4346,
217 180,1113,4655,3461,2028,1698, 833,2696,1235,1322,1594,4408,3623,3013,3225,2040,
218 3022, 541,2881, 607,3632,2029,1665,1219, 639,1385,1686,1099,2803,3231,1938,3188,
219 2858, 427, 676,2772,1168,2025, 454,3253,2486,3556, 230,1950, 580, 791,1991,1280,
220 1086,1974,2034, 630, 257,3338,2788,4903,1017, 86,4790, 966,2789,1995,1696,1131,
221 259,3095,4188,1308, 179,1463,5257, 289,4107,1248, 42,3413,1725,2288, 896,1947,
222 774,4474,4254, 604,3430,4264, 392,2514,2588, 452, 237,1408,3018, 988,4531,1970,
223 3034,3310, 540,2370,1562,1288,2990, 502,4765,1147, 4,1853,2708, 207, 294,2814,
224 4078,2902,2509, 684, 34,3105,3532,2551, 644, 709,2801,2344, 573,1727,3573,3557,
225 2021,1081,3100,4315,2100,3681, 199,2263,1837,2385, 146,3484,1195,2776,3949, 997,
226 1939,3973,1008,1091,1202,1962,1847,1149,4209,5444,1076, 493, 117,5400,2521, 972,
227 1490,2934,1796,4542,2374,1512,2933,2657, 413,2888,1135,2762,2314,2156,1355,2369,
228 766,2007,2527,2170,3124,2491,2593,2632,4757,2437, 234,3125,3591,1898,1750,1376,
229 1942,3468,3138, 570,2127,2145,3276,4131, 962, 132,1445,4196, 19, 941,3624,3480,
230 3366,1973,1374,4461,3431,2629, 283,2415,2275, 808,2887,3620,2112,2563,1353,3610,
231 955,1089,3103,1053, 96, 88,4097, 823,3808,1583, 399, 292,4091,3313, 421,1128,
232 642,4006, 903,2539,1877,2082, 596, 29,4066,1790, 722,2157, 130, 995,1569, 769,
233 1485, 464, 513,2213, 288,1923,1101,2453,4316, 133, 486,2445, 50, 625, 487,2207,
234 57, 423, 481,2962, 159,3729,1558, 491, 303, 482, 501, 240,2837, 112,3648,2392,
235 1783, 362, 8,3433,3422, 610,2793,3277,1390,1284,1654, 21,3823, 734, 367, 623,
236 193, 287, 374,1009,1483, 816, 476, 313,2255,2340,1262,2150,2899,1146,2581, 782,
237 2116,1659,2018,1880, 255,3586,3314,1110,2867,2137,2564, 986,2767,5185,2006, 650,
238 158, 926, 762, 881,3157,2717,2362,3587, 306,3690,3245,1542,3077,2427,1691,2478,
239 2118,2985,3490,2438, 539,2305, 983, 129,1754, 355,4201,2386, 827,2923, 104,1773,
240 2838,2771, 411,2905,3919, 376, 767, 122,1114, 828,2422,1817,3506, 266,3460,1007,
241 1609,4998, 945,2612,4429,2274, 726,1247,1964,2914,2199,2070,4002,4108, 657,3323,
242 1422, 579, 455,2764,4737,1222,2895,1670, 824,1223,1487,2525, 558, 861,3080, 598,
243 2659,2515,1967, 752,2583,2376,2214,4180, 977, 704,2464,4999,2622,4109,1210,2961,
244 819,1541, 142,2284, 44, 418, 457,1126,3730,4347,4626,1644,1876,3671,1864, 302,
245 1063,5694, 624, 723,1984,3745,1314,1676,2488,1610,1449,3558,3569,2166,2098, 409,
246 1011,2325,3704,2306, 818,1732,1383,1824,1844,3757, 999,2705,3497,1216,1423,2683,
247 2426,2954,2501,2726,2229,1475,2554,5064,1971,1794,1666,2014,1343, 783, 724, 191,
248 2434,1354,2220,5065,1763,2752,2472,4152, 131, 175,2885,3434, 92,1466,4920,2616,
249 3871,3872,3866, 128,1551,1632, 669,1854,3682,4691,4125,1230, 188,2973,3290,1302,
250 1213, 560,3266, 917, 763,3909,3249,1760, 868,1958, 764,1782,2097, 145,2277,3774,
251 4462, 64,1491,3062, 971,2132,3606,2442, 221,1226,1617, 218, 323,1185,3207,3147,
252 571, 619,1473,1005,1744,2281, 449,1887,2396,3685, 275, 375,3816,1743,3844,3731,
253 845,1983,2350,4210,1377, 773, 967,3499,3052,3743,2725,4007,1697,1022,3943,1464,
254 3264,2855,2722,1952,1029,2839,2467, 84,4383,2215, 820,1391,2015,2448,3672, 377,
255 1948,2168, 797,2545,3536,2578,2645, 94,2874,1678, 405,1259,3071, 771, 546,1315,
256 470,1243,3083, 895,2468, 981, 969,2037, 846,4181, 653,1276,2928, 14,2594, 557,
257 3007,2474, 156, 902,1338,1740,2574, 537,2518, 973,2282,2216,2433,1928, 138,2903,
258 1293,2631,1612, 646,3457, 839,2935, 111, 496,2191,2847, 589,3186, 149,3994,2060,
259 4031,2641,4067,3145,1870, 37,3597,2136,1025,2051,3009,3383,3549,1121,1016,3261,
260 1301, 251,2446,2599,2153, 872,3246, 637, 334,3705, 831, 884, 921,3065,3140,4092,
261 2198,1944, 246,2964, 108,2045,1152,1921,2308,1031, 203,3173,4170,1907,3890, 810,
262 1401,2003,1690, 506, 647,1242,2828,1761,1649,3208,2249,1589,3709,2931,5156,1708,
263 498, 666,2613, 834,3817,1231, 184,2851,1124, 883,3197,2261,3710,1765,1553,2658,
264 1178,2639,2351, 93,1193, 942,2538,2141,4402, 235,1821, 870,1591,2192,1709,1871,
265 3341,1618,4126,2595,2334, 603, 651, 69, 701, 268,2662,3411,2555,1380,1606, 503,
266 448, 254,2371,2646, 574,1187,2309,1770, 322,2235,1292,1801, 305, 566,1133, 229,
267 2067,2057, 706, 167, 483,2002,2672,3295,1820,3561,3067, 316, 378,2746,3452,1112,
268 136,1981, 507,1651,2917,1117, 285,4591, 182,2580,3522,1304, 335,3303,1835,2504,
269 1795,1792,2248, 674,1018,2106,2449,1857,2292,2845, 976,3047,1781,2600,2727,1389,
270 1281, 52,3152, 153, 265,3950, 672,3485,3951,4463, 430,1183, 365, 278,2169, 27,
271 1407,1336,2304, 209,1340,1730,2202,1852,2403,2883, 979,1737,1062, 631,2829,2542,
272 3876,2592, 825,2086,2226,3048,3625, 352,1417,3724, 542, 991, 431,1351,3938,1861,
273 2294, 826,1361,2927,3142,3503,1738, 463,2462,2723, 582,1916,1595,2808, 400,3845,
274 3891,2868,3621,2254, 58,2492,1123, 910,2160,2614,1372,1603,1196,1072,3385,1700,
275 3267,1980, 696, 480,2430, 920, 799,1570,2920,1951,2041,4047,2540,1321,4223,2469,
276 3562,2228,1271,2602, 401,2833,3351,2575,5157, 907,2312,1256, 410, 263,3507,1582,
277 996, 678,1849,2316,1480, 908,3545,2237, 703,2322, 667,1826,2849,1531,2604,2999,
278 2407,3146,2151,2630,1786,3711, 469,3542, 497,3899,2409, 858, 837,4446,3393,1274,
279 786, 620,1845,2001,3311, 484, 308,3367,1204,1815,3691,2332,1532,2557,1842,2020,
280 2724,1927,2333,4440, 567, 22,1673,2728,4475,1987,1858,1144,1597, 101,1832,3601,
281 12, 974,3783,4391, 951,1412, 1,3720, 453,4608,4041, 528,1041,1027,3230,2628,
282 1129, 875,1051,3291,1203,2262,1069,2860,2799,2149,2615,3278, 144,1758,3040, 31,
283 475,1680, 366,2685,3184, 311,1642,4008,2466,5036,1593,1493,2809, 216,1420,1668,
284 233, 304,2128,3284, 232,1429,1768,1040,2008,3407,2740,2967,2543, 242,2133, 778,
285 1565,2022,2620, 505,2189,2756,1098,2273, 372,1614, 708, 553,2846,2094,2278, 169,
286 3626,2835,4161, 228,2674,3165, 809,1454,1309, 466,1705,1095, 900,3423, 880,2667,
287 3751,5258,2317,3109,2571,4317,2766,1503,1342, 866,4447,1118, 63,2076, 314,1881,
288 1348,1061, 172, 978,3515,1747, 532, 511,3970, 6, 601, 905,2699,3300,1751, 276,
289 1467,3725,2668, 65,4239,2544,2779,2556,1604, 578,2451,1802, 992,2331,2624,1320,
290 3446, 713,1513,1013, 103,2786,2447,1661, 886,1702, 916, 654,3574,2031,1556, 751,
291 2178,2821,2179,1498,1538,2176, 271, 914,2251,2080,1325, 638,1953,2937,3877,2432,
292 2754, 95,3265,1716, 260,1227,4083, 775, 106,1357,3254, 426,1607, 555,2480, 772,
293 1985, 244,2546, 474, 495,1046,2611,1851,2061, 71,2089,1675,2590, 742,3758,2843,
294 3222,1433, 267,2180,2576,2826,2233,2092,3913,2435, 956,1745,3075, 856,2113,1116,
295 451, 3,1988,2896,1398, 993,2463,1878,2049,1341,2718,2721,2870,2108, 712,2904,
296 4363,2753,2324, 277,2872,2349,2649, 384, 987, 435, 691,3000, 922, 164,3939, 652,
297 1500,1184,4153,2482,3373,2165,4848,2335,3775,3508,3154,2806,2830,1554,2102,1664,
298 2530,1434,2408, 893,1547,2623,3447,2832,2242,2532,3169,2856,3223,2078, 49,3770,
299 3469, 462, 318, 656,2259,3250,3069, 679,1629,2758, 344,1138,1104,3120,1836,1283,
300 3115,2154,1437,4448, 934, 759,1999, 794,2862,1038, 533,2560,1722,2342, 855,2626,
301 1197,1663,4476,3127, 85,4240,2528, 25,1111,1181,3673, 407,3470,4561,2679,2713,
302 768,1925,2841,3986,1544,1165, 932, 373,1240,2146,1930,2673, 721,4766, 354,4333,
303 391,2963, 187, 61,3364,1442,1102, 330,1940,1767, 341,3809,4118, 393,2496,2062,
304 2211, 105, 331, 300, 439, 913,1332, 626, 379,3304,1557, 328, 689,3952, 309,1555,
305 931, 317,2517,3027, 325, 569, 686,2107,3084, 60,1042,1333,2794, 264,3177,4014,
306 1628, 258,3712, 7,4464,1176,1043,1778, 683, 114,1975, 78,1492, 383,1886, 510,
307 386, 645,5291,2891,2069,3305,4138,3867,2939,2603,2493,1935,1066,1848,3588,1015,
308 1282,1289,4609, 697,1453,3044,2666,3611,1856,2412, 54, 719,1330, 568,3778,2459,
309 1748, 788, 492, 551,1191,1000, 488,3394,3763, 282,1799, 348,2016,1523,3155,2390,
310 1049, 382,2019,1788,1170, 729,2968,3523, 897,3926,2785,2938,3292, 350,2319,3238,
311 1718,1717,2655,3453,3143,4465, 161,2889,2980,2009,1421, 56,1908,1640,2387,2232,
312 1917,1874,2477,4921, 148, 83,3438, 592,4245,2882,1822,1055, 741, 115,1496,1624,
313 381,1638,4592,1020, 516,3214, 458, 947,4575,1432, 211,1514,2926,1865,2142, 189,
314 852,1221,1400,1486, 882,2299,4036, 351, 28,1122, 700,6479,6480,6481,6482,6483, //last 512
315
316 /***************************************************************************************
317 *Everything below is of no interest for detection purpose *
318 ***************************************************************************************
319
320 5508,6484,3900,3414,3974,4441,4024,3537,4037,5628,5099,3633,6485,3148,6486,3636,
321 5509,3257,5510,5973,5445,5872,4941,4403,3174,4627,5873,6276,2286,4230,5446,5874,
322 5122,6102,6103,4162,5447,5123,5323,4849,6277,3980,3851,5066,4246,5774,5067,6278,
323 3001,2807,5695,3346,5775,5974,5158,5448,6487,5975,5976,5776,3598,6279,5696,4806,
324 4211,4154,6280,6488,6489,6490,6281,4212,5037,3374,4171,6491,4562,4807,4722,4827,
325 5977,6104,4532,4079,5159,5324,5160,4404,3858,5359,5875,3975,4288,4610,3486,4512,
326 5325,3893,5360,6282,6283,5560,2522,4231,5978,5186,5449,2569,3878,6284,5401,3578,
327 4415,6285,4656,5124,5979,2506,4247,4449,3219,3417,4334,4969,4329,6492,4576,4828,
328 4172,4416,4829,5402,6286,3927,3852,5361,4369,4830,4477,4867,5876,4173,6493,6105,
329 4657,6287,6106,5877,5450,6494,4155,4868,5451,3700,5629,4384,6288,6289,5878,3189,
330 4881,6107,6290,6495,4513,6496,4692,4515,4723,5100,3356,6497,6291,3810,4080,5561,
331 3570,4430,5980,6498,4355,5697,6499,4724,6108,6109,3764,4050,5038,5879,4093,3226,
332 6292,5068,5217,4693,3342,5630,3504,4831,4377,4466,4309,5698,4431,5777,6293,5778,
333 4272,3706,6110,5326,3752,4676,5327,4273,5403,4767,5631,6500,5699,5880,3475,5039,
334 6294,5562,5125,4348,4301,4482,4068,5126,4593,5700,3380,3462,5981,5563,3824,5404,
335 4970,5511,3825,4738,6295,6501,5452,4516,6111,5881,5564,6502,6296,5982,6503,4213,
336 4163,3454,6504,6112,4009,4450,6113,4658,6297,6114,3035,6505,6115,3995,4904,4739,
337 4563,4942,4110,5040,3661,3928,5362,3674,6506,5292,3612,4791,5565,4149,5983,5328,
338 5259,5021,4725,4577,4564,4517,4364,6298,5405,4578,5260,4594,4156,4157,5453,3592,
339 3491,6507,5127,5512,4709,4922,5984,5701,4726,4289,6508,4015,6116,5128,4628,3424,
340 4241,5779,6299,4905,6509,6510,5454,5702,5780,6300,4365,4923,3971,6511,5161,3270,
341 3158,5985,4100, 867,5129,5703,6117,5363,3695,3301,5513,4467,6118,6512,5455,4232,
342 4242,4629,6513,3959,4478,6514,5514,5329,5986,4850,5162,5566,3846,4694,6119,5456,
343 4869,5781,3779,6301,5704,5987,5515,4710,6302,5882,6120,4392,5364,5705,6515,6121,
344 6516,6517,3736,5988,5457,5989,4695,2457,5883,4551,5782,6303,6304,6305,5130,4971,
345 6122,5163,6123,4870,3263,5365,3150,4871,6518,6306,5783,5069,5706,3513,3498,4409,
346 5330,5632,5366,5458,5459,3991,5990,4502,3324,5991,5784,3696,4518,5633,4119,6519,
347 4630,5634,4417,5707,4832,5992,3418,6124,5993,5567,4768,5218,6520,4595,3458,5367,
348 6125,5635,6126,4202,6521,4740,4924,6307,3981,4069,4385,6308,3883,2675,4051,3834,
349 4302,4483,5568,5994,4972,4101,5368,6309,5164,5884,3922,6127,6522,6523,5261,5460,
350 5187,4164,5219,3538,5516,4111,3524,5995,6310,6311,5369,3181,3386,2484,5188,3464,
351 5569,3627,5708,6524,5406,5165,4677,4492,6312,4872,4851,5885,4468,5996,6313,5709,
352 5710,6128,2470,5886,6314,5293,4882,5785,3325,5461,5101,6129,5711,5786,6525,4906,
353 6526,6527,4418,5887,5712,4808,2907,3701,5713,5888,6528,3765,5636,5331,6529,6530,
354 3593,5889,3637,4943,3692,5714,5787,4925,6315,6130,5462,4405,6131,6132,6316,5262,
355 6531,6532,5715,3859,5716,5070,4696,5102,3929,5788,3987,4792,5997,6533,6534,3920,
356 4809,5000,5998,6535,2974,5370,6317,5189,5263,5717,3826,6536,3953,5001,4883,3190,
357 5463,5890,4973,5999,4741,6133,6134,3607,5570,6000,4711,3362,3630,4552,5041,6318,
358 6001,2950,2953,5637,4646,5371,4944,6002,2044,4120,3429,6319,6537,5103,4833,6538,
359 6539,4884,4647,3884,6003,6004,4758,3835,5220,5789,4565,5407,6540,6135,5294,4697,
360 4852,6320,6321,3206,4907,6541,6322,4945,6542,6136,6543,6323,6005,4631,3519,6544,
361 5891,6545,5464,3784,5221,6546,5571,4659,6547,6324,6137,5190,6548,3853,6549,4016,
362 4834,3954,6138,5332,3827,4017,3210,3546,4469,5408,5718,3505,4648,5790,5131,5638,
363 5791,5465,4727,4318,6325,6326,5792,4553,4010,4698,3439,4974,3638,4335,3085,6006,
364 5104,5042,5166,5892,5572,6327,4356,4519,5222,5573,5333,5793,5043,6550,5639,5071,
365 4503,6328,6139,6551,6140,3914,3901,5372,6007,5640,4728,4793,3976,3836,4885,6552,
366 4127,6553,4451,4102,5002,6554,3686,5105,6555,5191,5072,5295,4611,5794,5296,6556,
367 5893,5264,5894,4975,5466,5265,4699,4976,4370,4056,3492,5044,4886,6557,5795,4432,
368 4769,4357,5467,3940,4660,4290,6141,4484,4770,4661,3992,6329,4025,4662,5022,4632,
369 4835,4070,5297,4663,4596,5574,5132,5409,5895,6142,4504,5192,4664,5796,5896,3885,
370 5575,5797,5023,4810,5798,3732,5223,4712,5298,4084,5334,5468,6143,4052,4053,4336,
371 4977,4794,6558,5335,4908,5576,5224,4233,5024,4128,5469,5225,4873,6008,5045,4729,
372 4742,4633,3675,4597,6559,5897,5133,5577,5003,5641,5719,6330,6560,3017,2382,3854,
373 4406,4811,6331,4393,3964,4946,6561,2420,3722,6562,4926,4378,3247,1736,4442,6332,
374 5134,6333,5226,3996,2918,5470,4319,4003,4598,4743,4744,4485,3785,3902,5167,5004,
375 5373,4394,5898,6144,4874,1793,3997,6334,4085,4214,5106,5642,4909,5799,6009,4419,
376 4189,3330,5899,4165,4420,5299,5720,5227,3347,6145,4081,6335,2876,3930,6146,3293,
377 3786,3910,3998,5900,5300,5578,2840,6563,5901,5579,6147,3531,5374,6564,6565,5580,
378 4759,5375,6566,6148,3559,5643,6336,6010,5517,6337,6338,5721,5902,3873,6011,6339,
379 6567,5518,3868,3649,5722,6568,4771,4947,6569,6149,4812,6570,2853,5471,6340,6341,
380 5644,4795,6342,6012,5723,6343,5724,6013,4349,6344,3160,6150,5193,4599,4514,4493,
381 5168,4320,6345,4927,3666,4745,5169,5903,5005,4928,6346,5725,6014,4730,4203,5046,
382 4948,3395,5170,6015,4150,6016,5726,5519,6347,5047,3550,6151,6348,4197,4310,5904,
383 6571,5581,2965,6152,4978,3960,4291,5135,6572,5301,5727,4129,4026,5905,4853,5728,
384 5472,6153,6349,4533,2700,4505,5336,4678,3583,5073,2994,4486,3043,4554,5520,6350,
385 6017,5800,4487,6351,3931,4103,5376,6352,4011,4321,4311,4190,5136,6018,3988,3233,
386 4350,5906,5645,4198,6573,5107,3432,4191,3435,5582,6574,4139,5410,6353,5411,3944,
387 5583,5074,3198,6575,6354,4358,6576,5302,4600,5584,5194,5412,6577,6578,5585,5413,
388 5303,4248,5414,3879,4433,6579,4479,5025,4854,5415,6355,4760,4772,3683,2978,4700,
389 3797,4452,3965,3932,3721,4910,5801,6580,5195,3551,5907,3221,3471,3029,6019,3999,
390 5908,5909,5266,5267,3444,3023,3828,3170,4796,5646,4979,4259,6356,5647,5337,3694,
391 6357,5648,5338,4520,4322,5802,3031,3759,4071,6020,5586,4836,4386,5048,6581,3571,
392 4679,4174,4949,6154,4813,3787,3402,3822,3958,3215,3552,5268,4387,3933,4950,4359,
393 6021,5910,5075,3579,6358,4234,4566,5521,6359,3613,5049,6022,5911,3375,3702,3178,
394 4911,5339,4521,6582,6583,4395,3087,3811,5377,6023,6360,6155,4027,5171,5649,4421,
395 4249,2804,6584,2270,6585,4000,4235,3045,6156,5137,5729,4140,4312,3886,6361,4330,
396 6157,4215,6158,3500,3676,4929,4331,3713,4930,5912,4265,3776,3368,5587,4470,4855,
397 3038,4980,3631,6159,6160,4132,4680,6161,6362,3923,4379,5588,4255,6586,4121,6587,
398 6363,4649,6364,3288,4773,4774,6162,6024,6365,3543,6588,4274,3107,3737,5050,5803,
399 4797,4522,5589,5051,5730,3714,4887,5378,4001,4523,6163,5026,5522,4701,4175,2791,
400 3760,6589,5473,4224,4133,3847,4814,4815,4775,3259,5416,6590,2738,6164,6025,5304,
401 3733,5076,5650,4816,5590,6591,6165,6592,3934,5269,6593,3396,5340,6594,5804,3445,
402 3602,4042,4488,5731,5732,3525,5591,4601,5196,6166,6026,5172,3642,4612,3202,4506,
403 4798,6366,3818,5108,4303,5138,5139,4776,3332,4304,2915,3415,4434,5077,5109,4856,
404 2879,5305,4817,6595,5913,3104,3144,3903,4634,5341,3133,5110,5651,5805,6167,4057,
405 5592,2945,4371,5593,6596,3474,4182,6367,6597,6168,4507,4279,6598,2822,6599,4777,
406 4713,5594,3829,6169,3887,5417,6170,3653,5474,6368,4216,2971,5228,3790,4579,6369,
407 5733,6600,6601,4951,4746,4555,6602,5418,5475,6027,3400,4665,5806,6171,4799,6028,
408 5052,6172,3343,4800,4747,5006,6370,4556,4217,5476,4396,5229,5379,5477,3839,5914,
409 5652,5807,4714,3068,4635,5808,6173,5342,4192,5078,5419,5523,5734,6174,4557,6175,
410 4602,6371,6176,6603,5809,6372,5735,4260,3869,5111,5230,6029,5112,6177,3126,4681,
411 5524,5915,2706,3563,4748,3130,6178,4018,5525,6604,6605,5478,4012,4837,6606,4534,
412 4193,5810,4857,3615,5479,6030,4082,3697,3539,4086,5270,3662,4508,4931,5916,4912,
413 5811,5027,3888,6607,4397,3527,3302,3798,2775,2921,2637,3966,4122,4388,4028,4054,
414 1633,4858,5079,3024,5007,3982,3412,5736,6608,3426,3236,5595,3030,6179,3427,3336,
415 3279,3110,6373,3874,3039,5080,5917,5140,4489,3119,6374,5812,3405,4494,6031,4666,
416 4141,6180,4166,6032,5813,4981,6609,5081,4422,4982,4112,3915,5653,3296,3983,6375,
417 4266,4410,5654,6610,6181,3436,5082,6611,5380,6033,3819,5596,4535,5231,5306,5113,
418 6612,4952,5918,4275,3113,6613,6376,6182,6183,5814,3073,4731,4838,5008,3831,6614,
419 4888,3090,3848,4280,5526,5232,3014,5655,5009,5737,5420,5527,6615,5815,5343,5173,
420 5381,4818,6616,3151,4953,6617,5738,2796,3204,4360,2989,4281,5739,5174,5421,5197,
421 3132,5141,3849,5142,5528,5083,3799,3904,4839,5480,2880,4495,3448,6377,6184,5271,
422 5919,3771,3193,6034,6035,5920,5010,6036,5597,6037,6378,6038,3106,5422,6618,5423,
423 5424,4142,6619,4889,5084,4890,4313,5740,6620,3437,5175,5307,5816,4199,5198,5529,
424 5817,5199,5656,4913,5028,5344,3850,6185,2955,5272,5011,5818,4567,4580,5029,5921,
425 3616,5233,6621,6622,6186,4176,6039,6379,6380,3352,5200,5273,2908,5598,5234,3837,
426 5308,6623,6624,5819,4496,4323,5309,5201,6625,6626,4983,3194,3838,4167,5530,5922,
427 5274,6381,6382,3860,3861,5599,3333,4292,4509,6383,3553,5481,5820,5531,4778,6187,
428 3955,3956,4324,4389,4218,3945,4325,3397,2681,5923,4779,5085,4019,5482,4891,5382,
429 5383,6040,4682,3425,5275,4094,6627,5310,3015,5483,5657,4398,5924,3168,4819,6628,
430 5925,6629,5532,4932,4613,6041,6630,4636,6384,4780,4204,5658,4423,5821,3989,4683,
431 5822,6385,4954,6631,5345,6188,5425,5012,5384,3894,6386,4490,4104,6632,5741,5053,
432 6633,5823,5926,5659,5660,5927,6634,5235,5742,5824,4840,4933,4820,6387,4859,5928,
433 4955,6388,4143,3584,5825,5346,5013,6635,5661,6389,5014,5484,5743,4337,5176,5662,
434 6390,2836,6391,3268,6392,6636,6042,5236,6637,4158,6638,5744,5663,4471,5347,3663,
435 4123,5143,4293,3895,6639,6640,5311,5929,5826,3800,6189,6393,6190,5664,5348,3554,
436 3594,4749,4603,6641,5385,4801,6043,5827,4183,6642,5312,5426,4761,6394,5665,6191,
437 4715,2669,6643,6644,5533,3185,5427,5086,5930,5931,5386,6192,6044,6645,4781,4013,
438 5745,4282,4435,5534,4390,4267,6045,5746,4984,6046,2743,6193,3501,4087,5485,5932,
439 5428,4184,4095,5747,4061,5054,3058,3862,5933,5600,6646,5144,3618,6395,3131,5055,
440 5313,6396,4650,4956,3855,6194,3896,5202,4985,4029,4225,6195,6647,5828,5486,5829,
441 3589,3002,6648,6397,4782,5276,6649,6196,6650,4105,3803,4043,5237,5830,6398,4096,
442 3643,6399,3528,6651,4453,3315,4637,6652,3984,6197,5535,3182,3339,6653,3096,2660,
443 6400,6654,3449,5934,4250,4236,6047,6401,5831,6655,5487,3753,4062,5832,6198,6199,
444 6656,3766,6657,3403,4667,6048,6658,4338,2897,5833,3880,2797,3780,4326,6659,5748,
445 5015,6660,5387,4351,5601,4411,6661,3654,4424,5935,4339,4072,5277,4568,5536,6402,
446 6662,5238,6663,5349,5203,6200,5204,6201,5145,4536,5016,5056,4762,5834,4399,4957,
447 6202,6403,5666,5749,6664,4340,6665,5936,5177,5667,6666,6667,3459,4668,6404,6668,
448 6669,4543,6203,6670,4276,6405,4480,5537,6671,4614,5205,5668,6672,3348,2193,4763,
449 6406,6204,5937,5602,4177,5669,3419,6673,4020,6205,4443,4569,5388,3715,3639,6407,
450 6049,4058,6206,6674,5938,4544,6050,4185,4294,4841,4651,4615,5488,6207,6408,6051,
451 5178,3241,3509,5835,6208,4958,5836,4341,5489,5278,6209,2823,5538,5350,5206,5429,
452 6675,4638,4875,4073,3516,4684,4914,4860,5939,5603,5389,6052,5057,3237,5490,3791,
453 6676,6409,6677,4821,4915,4106,5351,5058,4243,5539,4244,5604,4842,4916,5239,3028,
454 3716,5837,5114,5605,5390,5940,5430,6210,4332,6678,5540,4732,3667,3840,6053,4305,
455 3408,5670,5541,6410,2744,5240,5750,6679,3234,5606,6680,5607,5671,3608,4283,4159,
456 4400,5352,4783,6681,6411,6682,4491,4802,6211,6412,5941,6413,6414,5542,5751,6683,
457 4669,3734,5942,6684,6415,5943,5059,3328,4670,4144,4268,6685,6686,6687,6688,4372,
458 3603,6689,5944,5491,4373,3440,6416,5543,4784,4822,5608,3792,4616,5838,5672,3514,
459 5391,6417,4892,6690,4639,6691,6054,5673,5839,6055,6692,6056,5392,6212,4038,5544,
460 5674,4497,6057,6693,5840,4284,5675,4021,4545,5609,6418,4454,6419,6213,4113,4472,
461 5314,3738,5087,5279,4074,5610,4959,4063,3179,4750,6058,6420,6214,3476,4498,4716,
462 5431,4960,4685,6215,5241,6694,6421,6216,6695,5841,5945,6422,3748,5946,5179,3905,
463 5752,5545,5947,4374,6217,4455,6423,4412,6218,4803,5353,6696,3832,5280,6219,4327,
464 4702,6220,6221,6059,4652,5432,6424,3749,4751,6425,5753,4986,5393,4917,5948,5030,
465 5754,4861,4733,6426,4703,6697,6222,4671,5949,4546,4961,5180,6223,5031,3316,5281,
466 6698,4862,4295,4934,5207,3644,6427,5842,5950,6428,6429,4570,5843,5282,6430,6224,
467 5088,3239,6060,6699,5844,5755,6061,6431,2701,5546,6432,5115,5676,4039,3993,3327,
468 4752,4425,5315,6433,3941,6434,5677,4617,4604,3074,4581,6225,5433,6435,6226,6062,
469 4823,5756,5116,6227,3717,5678,4717,5845,6436,5679,5846,6063,5847,6064,3977,3354,
470 6437,3863,5117,6228,5547,5394,4499,4524,6229,4605,6230,4306,4500,6700,5951,6065,
471 3693,5952,5089,4366,4918,6701,6231,5548,6232,6702,6438,4704,5434,6703,6704,5953,
472 4168,6705,5680,3420,6706,5242,4407,6066,3812,5757,5090,5954,4672,4525,3481,5681,
473 4618,5395,5354,5316,5955,6439,4962,6707,4526,6440,3465,4673,6067,6441,5682,6708,
474 5435,5492,5758,5683,4619,4571,4674,4804,4893,4686,5493,4753,6233,6068,4269,6442,
475 6234,5032,4705,5146,5243,5208,5848,6235,6443,4963,5033,4640,4226,6236,5849,3387,
476 6444,6445,4436,4437,5850,4843,5494,4785,4894,6709,4361,6710,5091,5956,3331,6237,
477 4987,5549,6069,6711,4342,3517,4473,5317,6070,6712,6071,4706,6446,5017,5355,6713,
478 6714,4988,5436,6447,4734,5759,6715,4735,4547,4456,4754,6448,5851,6449,6450,3547,
479 5852,5318,6451,6452,5092,4205,6716,6238,4620,4219,5611,6239,6072,4481,5760,5957,
480 5958,4059,6240,6453,4227,4537,6241,5761,4030,4186,5244,5209,3761,4457,4876,3337,
481 5495,5181,6242,5959,5319,5612,5684,5853,3493,5854,6073,4169,5613,5147,4895,6074,
482 5210,6717,5182,6718,3830,6243,2798,3841,6075,6244,5855,5614,3604,4606,5496,5685,
483 5118,5356,6719,6454,5960,5357,5961,6720,4145,3935,4621,5119,5962,4261,6721,6455,
484 4786,5963,4375,4582,6245,6246,6247,6076,5437,4877,5856,3376,4380,6248,4160,6722,
485 5148,6456,5211,6457,6723,4718,6458,6724,6249,5358,4044,3297,6459,6250,5857,5615,
486 5497,5245,6460,5498,6725,6251,6252,5550,3793,5499,2959,5396,6461,6462,4572,5093,
487 5500,5964,3806,4146,6463,4426,5762,5858,6077,6253,4755,3967,4220,5965,6254,4989,
488 5501,6464,4352,6726,6078,4764,2290,5246,3906,5438,5283,3767,4964,2861,5763,5094,
489 6255,6256,4622,5616,5859,5860,4707,6727,4285,4708,4824,5617,6257,5551,4787,5212,
490 4965,4935,4687,6465,6728,6466,5686,6079,3494,4413,2995,5247,5966,5618,6729,5967,
491 5764,5765,5687,5502,6730,6731,6080,5397,6467,4990,6258,6732,4538,5060,5619,6733,
492 4719,5688,5439,5018,5149,5284,5503,6734,6081,4607,6259,5120,3645,5861,4583,6260,
493 4584,4675,5620,4098,5440,6261,4863,2379,3306,4585,5552,5689,4586,5285,6735,4864,
494 6736,5286,6082,6737,4623,3010,4788,4381,4558,5621,4587,4896,3698,3161,5248,4353,
495 4045,6262,3754,5183,4588,6738,6263,6739,6740,5622,3936,6741,6468,6742,6264,5095,
496 6469,4991,5968,6743,4992,6744,6083,4897,6745,4256,5766,4307,3108,3968,4444,5287,
497 3889,4343,6084,4510,6085,4559,6086,4898,5969,6746,5623,5061,4919,5249,5250,5504,
498 5441,6265,5320,4878,3242,5862,5251,3428,6087,6747,4237,5624,5442,6266,5553,4539,
499 6748,2585,3533,5398,4262,6088,5150,4736,4438,6089,6267,5505,4966,6749,6268,6750,
500 6269,5288,5554,3650,6090,6091,4624,6092,5690,6751,5863,4270,5691,4277,5555,5864,
501 6752,5692,4720,4865,6470,5151,4688,4825,6753,3094,6754,6471,3235,4653,6755,5213,
502 5399,6756,3201,4589,5865,4967,6472,5866,6473,5019,3016,6757,5321,4756,3957,4573,
503 6093,4993,5767,4721,6474,6758,5625,6759,4458,6475,6270,6760,5556,4994,5214,5252,
504 6271,3875,5768,6094,5034,5506,4376,5769,6761,2120,6476,5253,5770,6762,5771,5970,
505 3990,5971,5557,5558,5772,6477,6095,2787,4641,5972,5121,6096,6097,6272,6763,3703,
506 5867,5507,6273,4206,6274,4789,6098,6764,3619,3646,3833,3804,2394,3788,4936,3978,
507 4866,4899,6099,6100,5559,6478,6765,3599,5868,6101,5869,5870,6275,6766,4527,6767,
508 *******************************************************************************/
509 };
510 }
+0
-587
src/org/mozilla/universalchardet/prober/distributionanalysis/JISDistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public abstract class JISDistributionAnalysis extends CharDistributionAnalysis
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final float JIS_TYPICAL_DISTRIBUTION_RATIO = 3.0f;
45
46
47 ////////////////////////////////////////////////////////////////
48 // methods
49 ////////////////////////////////////////////////////////////////
50 public JISDistributionAnalysis()
51 {
52 super();
53 this.typicalDistributionRatio = JIS_TYPICAL_DISTRIBUTION_RATIO;
54 this.charToFreqOrder = jisCharToFreqOrder;
55 }
56
57
58 ////////////////////////////////////////////////////////////////
59 // constants continued
60 ////////////////////////////////////////////////////////////////
61 private static final int[] jisCharToFreqOrder = new int[] {
62 40, 1, 6, 182, 152, 180, 295,2127, 285, 381,3295,4304,3068,4606,3165,3510, // 16
63 3511,1822,2785,4607,1193,2226,5070,4608, 171,2996,1247, 18, 179,5071, 856,1661, // 32
64 1262,5072, 619, 127,3431,3512,3230,1899,1700, 232, 228,1294,1298, 284, 283,2041, // 48
65 2042,1061,1062, 48, 49, 44, 45, 433, 434,1040,1041, 996, 787,2997,1255,4305, // 64
66 2108,4609,1684,1648,5073,5074,5075,5076,5077,5078,3687,5079,4610,5080,3927,3928, // 80
67 5081,3296,3432, 290,2285,1471,2187,5082,2580,2825,1303,2140,1739,1445,2691,3375, // 96
68 1691,3297,4306,4307,4611, 452,3376,1182,2713,3688,3069,4308,5083,5084,5085,5086, // 112
69 5087,5088,5089,5090,5091,5092,5093,5094,5095,5096,5097,5098,5099,5100,5101,5102, // 128
70 5103,5104,5105,5106,5107,5108,5109,5110,5111,5112,4097,5113,5114,5115,5116,5117, // 144
71 5118,5119,5120,5121,5122,5123,5124,5125,5126,5127,5128,5129,5130,5131,5132,5133, // 160
72 5134,5135,5136,5137,5138,5139,5140,5141,5142,5143,5144,5145,5146,5147,5148,5149, // 176
73 5150,5151,5152,4612,5153,5154,5155,5156,5157,5158,5159,5160,5161,5162,5163,5164, // 192
74 5165,5166,5167,5168,5169,5170,5171,5172,5173,5174,5175,1472, 598, 618, 820,1205, // 208
75 1309,1412,1858,1307,1692,5176,5177,5178,5179,5180,5181,5182,1142,1452,1234,1172, // 224
76 1875,2043,2149,1793,1382,2973, 925,2404,1067,1241, 960,1377,2935,1491, 919,1217, // 240
77 1865,2030,1406,1499,2749,4098,5183,5184,5185,5186,5187,5188,2561,4099,3117,1804, // 256
78 2049,3689,4309,3513,1663,5189,3166,3118,3298,1587,1561,3433,5190,3119,1625,2998, // 272
79 3299,4613,1766,3690,2786,4614,5191,5192,5193,5194,2161, 26,3377, 2,3929, 20, // 288
80 3691, 47,4100, 50, 17, 16, 35, 268, 27, 243, 42, 155, 24, 154, 29, 184, // 304
81 4, 91, 14, 92, 53, 396, 33, 289, 9, 37, 64, 620, 21, 39, 321, 5, // 320
82 12, 11, 52, 13, 3, 208, 138, 0, 7, 60, 526, 141, 151,1069, 181, 275, // 336
83 1591, 83, 132,1475, 126, 331, 829, 15, 69, 160, 59, 22, 157, 55,1079, 312, // 352
84 109, 38, 23, 25, 10, 19, 79,5195, 61, 382,1124, 8, 30,5196,5197,5198, // 368
85 5199,5200,5201,5202,5203,5204,5205,5206, 89, 62, 74, 34,2416, 112, 139, 196, // 384
86 271, 149, 84, 607, 131, 765, 46, 88, 153, 683, 76, 874, 101, 258, 57, 80, // 400
87 32, 364, 121,1508, 169,1547, 68, 235, 145,2999, 41, 360,3027, 70, 63, 31, // 416
88 43, 259, 262,1383, 99, 533, 194, 66, 93, 846, 217, 192, 56, 106, 58, 565, // 432
89 280, 272, 311, 256, 146, 82, 308, 71, 100, 128, 214, 655, 110, 261, 104,1140, // 448
90 54, 51, 36, 87, 67,3070, 185,2618,2936,2020, 28,1066,2390,2059,5207,5208, // 464
91 5209,5210,5211,5212,5213,5214,5215,5216,4615,5217,5218,5219,5220,5221,5222,5223, // 480
92 5224,5225,5226,5227,5228,5229,5230,5231,5232,5233,5234,5235,5236,3514,5237,5238, // 496
93 5239,5240,5241,5242,5243,5244,2297,2031,4616,4310,3692,5245,3071,5246,3598,5247, // 512
94 4617,3231,3515,5248,4101,4311,4618,3808,4312,4102,5249,4103,4104,3599,5250,5251, // 528
95 5252,5253,5254,5255,5256,5257,5258,5259,5260,5261,5262,5263,5264,5265,5266,5267, // 544
96 5268,5269,5270,5271,5272,5273,5274,5275,5276,5277,5278,5279,5280,5281,5282,5283, // 560
97 5284,5285,5286,5287,5288,5289,5290,5291,5292,5293,5294,5295,5296,5297,5298,5299, // 576
98 5300,5301,5302,5303,5304,5305,5306,5307,5308,5309,5310,5311,5312,5313,5314,5315, // 592
99 5316,5317,5318,5319,5320,5321,5322,5323,5324,5325,5326,5327,5328,5329,5330,5331, // 608
100 5332,5333,5334,5335,5336,5337,5338,5339,5340,5341,5342,5343,5344,5345,5346,5347, // 624
101 5348,5349,5350,5351,5352,5353,5354,5355,5356,5357,5358,5359,5360,5361,5362,5363, // 640
102 5364,5365,5366,5367,5368,5369,5370,5371,5372,5373,5374,5375,5376,5377,5378,5379, // 656
103 5380,5381, 363, 642,2787,2878,2788,2789,2316,3232,2317,3434,2011, 165,1942,3930, // 672
104 3931,3932,3933,5382,4619,5383,4620,5384,5385,5386,5387,5388,5389,5390,5391,5392, // 688
105 5393,5394,5395,5396,5397,5398,5399,5400,5401,5402,5403,5404,5405,5406,5407,5408, // 704
106 5409,5410,5411,5412,5413,5414,5415,5416,5417,5418,5419,5420,5421,5422,5423,5424, // 720
107 5425,5426,5427,5428,5429,5430,5431,5432,5433,5434,5435,5436,5437,5438,5439,5440, // 736
108 5441,5442,5443,5444,5445,5446,5447,5448,5449,5450,5451,5452,5453,5454,5455,5456, // 752
109 5457,5458,5459,5460,5461,5462,5463,5464,5465,5466,5467,5468,5469,5470,5471,5472, // 768
110 5473,5474,5475,5476,5477,5478,5479,5480,5481,5482,5483,5484,5485,5486,5487,5488, // 784
111 5489,5490,5491,5492,5493,5494,5495,5496,5497,5498,5499,5500,5501,5502,5503,5504, // 800
112 5505,5506,5507,5508,5509,5510,5511,5512,5513,5514,5515,5516,5517,5518,5519,5520, // 816
113 5521,5522,5523,5524,5525,5526,5527,5528,5529,5530,5531,5532,5533,5534,5535,5536, // 832
114 5537,5538,5539,5540,5541,5542,5543,5544,5545,5546,5547,5548,5549,5550,5551,5552, // 848
115 5553,5554,5555,5556,5557,5558,5559,5560,5561,5562,5563,5564,5565,5566,5567,5568, // 864
116 5569,5570,5571,5572,5573,5574,5575,5576,5577,5578,5579,5580,5581,5582,5583,5584, // 880
117 5585,5586,5587,5588,5589,5590,5591,5592,5593,5594,5595,5596,5597,5598,5599,5600, // 896
118 5601,5602,5603,5604,5605,5606,5607,5608,5609,5610,5611,5612,5613,5614,5615,5616, // 912
119 5617,5618,5619,5620,5621,5622,5623,5624,5625,5626,5627,5628,5629,5630,5631,5632, // 928
120 5633,5634,5635,5636,5637,5638,5639,5640,5641,5642,5643,5644,5645,5646,5647,5648, // 944
121 5649,5650,5651,5652,5653,5654,5655,5656,5657,5658,5659,5660,5661,5662,5663,5664, // 960
122 5665,5666,5667,5668,5669,5670,5671,5672,5673,5674,5675,5676,5677,5678,5679,5680, // 976
123 5681,5682,5683,5684,5685,5686,5687,5688,5689,5690,5691,5692,5693,5694,5695,5696, // 992
124 5697,5698,5699,5700,5701,5702,5703,5704,5705,5706,5707,5708,5709,5710,5711,5712, // 1008
125 5713,5714,5715,5716,5717,5718,5719,5720,5721,5722,5723,5724,5725,5726,5727,5728, // 1024
126 5729,5730,5731,5732,5733,5734,5735,5736,5737,5738,5739,5740,5741,5742,5743,5744, // 1040
127 5745,5746,5747,5748,5749,5750,5751,5752,5753,5754,5755,5756,5757,5758,5759,5760, // 1056
128 5761,5762,5763,5764,5765,5766,5767,5768,5769,5770,5771,5772,5773,5774,5775,5776, // 1072
129 5777,5778,5779,5780,5781,5782,5783,5784,5785,5786,5787,5788,5789,5790,5791,5792, // 1088
130 5793,5794,5795,5796,5797,5798,5799,5800,5801,5802,5803,5804,5805,5806,5807,5808, // 1104
131 5809,5810,5811,5812,5813,5814,5815,5816,5817,5818,5819,5820,5821,5822,5823,5824, // 1120
132 5825,5826,5827,5828,5829,5830,5831,5832,5833,5834,5835,5836,5837,5838,5839,5840, // 1136
133 5841,5842,5843,5844,5845,5846,5847,5848,5849,5850,5851,5852,5853,5854,5855,5856, // 1152
134 5857,5858,5859,5860,5861,5862,5863,5864,5865,5866,5867,5868,5869,5870,5871,5872, // 1168
135 5873,5874,5875,5876,5877,5878,5879,5880,5881,5882,5883,5884,5885,5886,5887,5888, // 1184
136 5889,5890,5891,5892,5893,5894,5895,5896,5897,5898,5899,5900,5901,5902,5903,5904, // 1200
137 5905,5906,5907,5908,5909,5910,5911,5912,5913,5914,5915,5916,5917,5918,5919,5920, // 1216
138 5921,5922,5923,5924,5925,5926,5927,5928,5929,5930,5931,5932,5933,5934,5935,5936, // 1232
139 5937,5938,5939,5940,5941,5942,5943,5944,5945,5946,5947,5948,5949,5950,5951,5952, // 1248
140 5953,5954,5955,5956,5957,5958,5959,5960,5961,5962,5963,5964,5965,5966,5967,5968, // 1264
141 5969,5970,5971,5972,5973,5974,5975,5976,5977,5978,5979,5980,5981,5982,5983,5984, // 1280
142 5985,5986,5987,5988,5989,5990,5991,5992,5993,5994,5995,5996,5997,5998,5999,6000, // 1296
143 6001,6002,6003,6004,6005,6006,6007,6008,6009,6010,6011,6012,6013,6014,6015,6016, // 1312
144 6017,6018,6019,6020,6021,6022,6023,6024,6025,6026,6027,6028,6029,6030,6031,6032, // 1328
145 6033,6034,6035,6036,6037,6038,6039,6040,6041,6042,6043,6044,6045,6046,6047,6048, // 1344
146 6049,6050,6051,6052,6053,6054,6055,6056,6057,6058,6059,6060,6061,6062,6063,6064, // 1360
147 6065,6066,6067,6068,6069,6070,6071,6072,6073,6074,6075,6076,6077,6078,6079,6080, // 1376
148 6081,6082,6083,6084,6085,6086,6087,6088,6089,6090,6091,6092,6093,6094,6095,6096, // 1392
149 6097,6098,6099,6100,6101,6102,6103,6104,6105,6106,6107,6108,6109,6110,6111,6112, // 1408
150 6113,6114,2044,2060,4621, 997,1235, 473,1186,4622, 920,3378,6115,6116, 379,1108, // 1424
151 4313,2657,2735,3934,6117,3809, 636,3233, 573,1026,3693,3435,2974,3300,2298,4105, // 1440
152 854,2937,2463, 393,2581,2417, 539, 752,1280,2750,2480, 140,1161, 440, 708,1569, // 1456
153 665,2497,1746,1291,1523,3000, 164,1603, 847,1331, 537,1997, 486, 508,1693,2418, // 1472
154 1970,2227, 878,1220, 299,1030, 969, 652,2751, 624,1137,3301,2619, 65,3302,2045, // 1488
155 1761,1859,3120,1930,3694,3516, 663,1767, 852, 835,3695, 269, 767,2826,2339,1305, // 1504
156 896,1150, 770,1616,6118, 506,1502,2075,1012,2519, 775,2520,2975,2340,2938,4314, // 1520
157 3028,2086,1224,1943,2286,6119,3072,4315,2240,1273,1987,3935,1557, 175, 597, 985, // 1536
158 3517,2419,2521,1416,3029, 585, 938,1931,1007,1052,1932,1685,6120,3379,4316,4623, // 1552
159 804, 599,3121,1333,2128,2539,1159,1554,2032,3810, 687,2033,2904, 952, 675,1467, // 1568
160 3436,6121,2241,1096,1786,2440,1543,1924, 980,1813,2228, 781,2692,1879, 728,1918, // 1584
161 3696,4624, 548,1950,4625,1809,1088,1356,3303,2522,1944, 502, 972, 373, 513,2827, // 1600
162 586,2377,2391,1003,1976,1631,6122,2464,1084, 648,1776,4626,2141, 324, 962,2012, // 1616
163 2177,2076,1384, 742,2178,1448,1173,1810, 222, 102, 301, 445, 125,2420, 662,2498, // 1632
164 277, 200,1476,1165,1068, 224,2562,1378,1446, 450,1880, 659, 791, 582,4627,2939, // 1648
165 3936,1516,1274, 555,2099,3697,1020,1389,1526,3380,1762,1723,1787,2229, 412,2114, // 1664
166 1900,2392,3518, 512,2597, 427,1925,2341,3122,1653,1686,2465,2499, 697, 330, 273, // 1680
167 380,2162, 951, 832, 780, 991,1301,3073, 965,2270,3519, 668,2523,2636,1286, 535, // 1696
168 1407, 518, 671, 957,2658,2378, 267, 611,2197,3030,6123, 248,2299, 967,1799,2356, // 1712
169 850,1418,3437,1876,1256,1480,2828,1718,6124,6125,1755,1664,2405,6126,4628,2879, // 1728
170 2829, 499,2179, 676,4629, 557,2329,2214,2090, 325,3234, 464, 811,3001, 992,2342, // 1744
171 2481,1232,1469, 303,2242, 466,1070,2163, 603,1777,2091,4630,2752,4631,2714, 322, // 1760
172 2659,1964,1768, 481,2188,1463,2330,2857,3600,2092,3031,2421,4632,2318,2070,1849, // 1776
173 2598,4633,1302,2254,1668,1701,2422,3811,2905,3032,3123,2046,4106,1763,1694,4634, // 1792
174 1604, 943,1724,1454, 917, 868,2215,1169,2940, 552,1145,1800,1228,1823,1955, 316, // 1808
175 1080,2510, 361,1807,2830,4107,2660,3381,1346,1423,1134,4108,6127, 541,1263,1229, // 1824
176 1148,2540, 545, 465,1833,2880,3438,1901,3074,2482, 816,3937, 713,1788,2500, 122, // 1840
177 1575, 195,1451,2501,1111,6128, 859, 374,1225,2243,2483,4317, 390,1033,3439,3075, // 1856
178 2524,1687, 266, 793,1440,2599, 946, 779, 802, 507, 897,1081, 528,2189,1292, 711, // 1872
179 1866,1725,1167,1640, 753, 398,2661,1053, 246, 348,4318, 137,1024,3440,1600,2077, // 1888
180 2129, 825,4319, 698, 238, 521, 187,2300,1157,2423,1641,1605,1464,1610,1097,2541, // 1904
181 1260,1436, 759,2255,1814,2150, 705,3235, 409,2563,3304, 561,3033,2005,2564, 726, // 1920
182 1956,2343,3698,4109, 949,3812,3813,3520,1669, 653,1379,2525, 881,2198, 632,2256, // 1936
183 1027, 778,1074, 733,1957, 514,1481,2466, 554,2180, 702,3938,1606,1017,1398,6129, // 1952
184 1380,3521, 921, 993,1313, 594, 449,1489,1617,1166, 768,1426,1360, 495,1794,3601, // 1968
185 1177,3602,1170,4320,2344, 476, 425,3167,4635,3168,1424, 401,2662,1171,3382,1998, // 1984
186 1089,4110, 477,3169, 474,6130,1909, 596,2831,1842, 494, 693,1051,1028,1207,3076, // 2000
187 606,2115, 727,2790,1473,1115, 743,3522, 630, 805,1532,4321,2021, 366,1057, 838, // 2016
188 684,1114,2142,4322,2050,1492,1892,1808,2271,3814,2424,1971,1447,1373,3305,1090, // 2032
189 1536,3939,3523,3306,1455,2199, 336, 369,2331,1035, 584,2393, 902, 718,2600,6131, // 2048
190 2753, 463,2151,1149,1611,2467, 715,1308,3124,1268, 343,1413,3236,1517,1347,2663, // 2064
191 2093,3940,2022,1131,1553,2100,2941,1427,3441,2942,1323,2484,6132,1980, 872,2368, // 2080
192 2441,2943, 320,2369,2116,1082, 679,1933,3941,2791,3815, 625,1143,2023, 422,2200, // 2096
193 3816,6133, 730,1695, 356,2257,1626,2301,2858,2637,1627,1778, 937, 883,2906,2693, // 2112
194 3002,1769,1086, 400,1063,1325,3307,2792,4111,3077, 456,2345,1046, 747,6134,1524, // 2128
195 884,1094,3383,1474,2164,1059, 974,1688,2181,2258,1047, 345,1665,1187, 358, 875, // 2144
196 3170, 305, 660,3524,2190,1334,1135,3171,1540,1649,2542,1527, 927, 968,2793, 885, // 2160
197 1972,1850, 482, 500,2638,1218,1109,1085,2543,1654,2034, 876, 78,2287,1482,1277, // 2176
198 861,1675,1083,1779, 724,2754, 454, 397,1132,1612,2332, 893, 672,1237, 257,2259, // 2192
199 2370, 135,3384, 337,2244, 547, 352, 340, 709,2485,1400, 788,1138,2511, 540, 772, // 2208
200 1682,2260,2272,2544,2013,1843,1902,4636,1999,1562,2288,4637,2201,1403,1533, 407, // 2224
201 576,3308,1254,2071, 978,3385, 170, 136,1201,3125,2664,3172,2394, 213, 912, 873, // 2240
202 3603,1713,2202, 699,3604,3699, 813,3442, 493, 531,1054, 468,2907,1483, 304, 281, // 2256
203 4112,1726,1252,2094, 339,2319,2130,2639, 756,1563,2944, 748, 571,2976,1588,2425, // 2272
204 2715,1851,1460,2426,1528,1392,1973,3237, 288,3309, 685,3386, 296, 892,2716,2216, // 2288
205 1570,2245, 722,1747,2217, 905,3238,1103,6135,1893,1441,1965, 251,1805,2371,3700, // 2304
206 2601,1919,1078, 75,2182,1509,1592,1270,2640,4638,2152,6136,3310,3817, 524, 706, // 2320
207 1075, 292,3818,1756,2602, 317, 98,3173,3605,3525,1844,2218,3819,2502, 814, 567, // 2336
208 385,2908,1534,6137, 534,1642,3239, 797,6138,1670,1529, 953,4323, 188,1071, 538, // 2352
209 178, 729,3240,2109,1226,1374,2000,2357,2977, 731,2468,1116,2014,2051,6139,1261, // 2368
210 1593, 803,2859,2736,3443, 556, 682, 823,1541,6140,1369,2289,1706,2794, 845, 462, // 2384
211 2603,2665,1361, 387, 162,2358,1740, 739,1770,1720,1304,1401,3241,1049, 627,1571, // 2400
212 2427,3526,1877,3942,1852,1500, 431,1910,1503, 677, 297,2795, 286,1433,1038,1198, // 2416
213 2290,1133,1596,4113,4639,2469,1510,1484,3943,6141,2442, 108, 712,4640,2372, 866, // 2432
214 3701,2755,3242,1348, 834,1945,1408,3527,2395,3243,1811, 824, 994,1179,2110,1548, // 2448
215 1453, 790,3003, 690,4324,4325,2832,2909,3820,1860,3821, 225,1748, 310, 346,1780, // 2464
216 2470, 821,1993,2717,2796, 828, 877,3528,2860,2471,1702,2165,2910,2486,1789, 453, // 2480
217 359,2291,1676, 73,1164,1461,1127,3311, 421, 604, 314,1037, 589, 116,2487, 737, // 2496
218 837,1180, 111, 244, 735,6142,2261,1861,1362, 986, 523, 418, 581,2666,3822, 103, // 2512
219 855, 503,1414,1867,2488,1091, 657,1597, 979, 605,1316,4641,1021,2443,2078,2001, // 2528
220 1209, 96, 587,2166,1032, 260,1072,2153, 173, 94, 226,3244, 819,2006,4642,4114, // 2544
221 2203, 231,1744, 782, 97,2667, 786,3387, 887, 391, 442,2219,4326,1425,6143,2694, // 2560
222 633,1544,1202, 483,2015, 592,2052,1958,2472,1655, 419, 129,4327,3444,3312,1714, // 2576
223 1257,3078,4328,1518,1098, 865,1310,1019,1885,1512,1734, 469,2444, 148, 773, 436, // 2592
224 1815,1868,1128,1055,4329,1245,2756,3445,2154,1934,1039,4643, 579,1238, 932,2320, // 2608
225 353, 205, 801, 115,2428, 944,2321,1881, 399,2565,1211, 678, 766,3944, 335,2101, // 2624
226 1459,1781,1402,3945,2737,2131,1010, 844, 981,1326,1013, 550,1816,1545,2620,1335, // 2640
227 1008, 371,2881, 936,1419,1613,3529,1456,1395,2273,1834,2604,1317,2738,2503, 416, // 2656
228 1643,4330, 806,1126, 229, 591,3946,1314,1981,1576,1837,1666, 347,1790, 977,3313, // 2672
229 764,2861,1853, 688,2429,1920,1462, 77, 595, 415,2002,3034, 798,1192,4115,6144, // 2688
230 2978,4331,3035,2695,2582,2072,2566, 430,2430,1727, 842,1396,3947,3702, 613, 377, // 2704
231 278, 236,1417,3388,3314,3174, 757,1869, 107,3530,6145,1194, 623,2262, 207,1253, // 2720
232 2167,3446,3948, 492,1117,1935, 536,1838,2757,1246,4332, 696,2095,2406,1393,1572, // 2736
233 3175,1782, 583, 190, 253,1390,2230, 830,3126,3389, 934,3245,1703,1749,2979,1870, // 2752
234 2545,1656,2204, 869,2346,4116,3176,1817, 496,1764,4644, 942,1504, 404,1903,1122, // 2768
235 1580,3606,2945,1022, 515, 372,1735, 955,2431,3036,6146,2797,1110,2302,2798, 617, // 2784
236 6147, 441, 762,1771,3447,3607,3608,1904, 840,3037, 86, 939,1385, 572,1370,2445, // 2800
237 1336, 114,3703, 898, 294, 203,3315, 703,1583,2274, 429, 961,4333,1854,1951,3390, // 2816
238 2373,3704,4334,1318,1381, 966,1911,2322,1006,1155, 309, 989, 458,2718,1795,1372, // 2832
239 1203, 252,1689,1363,3177, 517,1936, 168,1490, 562, 193,3823,1042,4117,1835, 551, // 2848
240 470,4645, 395, 489,3448,1871,1465,2583,2641, 417,1493, 279,1295, 511,1236,1119, // 2864
241 72,1231,1982,1812,3004, 871,1564, 984,3449,1667,2696,2096,4646,2347,2833,1673, // 2880
242 3609, 695,3246,2668, 807,1183,4647, 890, 388,2333,1801,1457,2911,1765,1477,1031, // 2896
243 3316,3317,1278,3391,2799,2292,2526, 163,3450,4335,2669,1404,1802,6148,2323,2407, // 2912
244 1584,1728,1494,1824,1269, 298, 909,3318,1034,1632, 375, 776,1683,2061, 291, 210, // 2928
245 1123, 809,1249,1002,2642,3038, 206,1011,2132, 144, 975, 882,1565, 342, 667, 754, // 2944
246 1442,2143,1299,2303,2062, 447, 626,2205,1221,2739,2912,1144,1214,2206,2584, 760, // 2960
247 1715, 614, 950,1281,2670,2621, 810, 577,1287,2546,4648, 242,2168, 250,2643, 691, // 2976
248 123,2644, 647, 313,1029, 689,1357,2946,1650, 216, 771,1339,1306, 808,2063, 549, // 2992
249 913,1371,2913,2914,6149,1466,1092,1174,1196,1311,2605,2396,1783,1796,3079, 406, // 3008
250 2671,2117,3949,4649, 487,1825,2220,6150,2915, 448,2348,1073,6151,2397,1707, 130, // 3024
251 900,1598, 329, 176,1959,2527,1620,6152,2275,4336,3319,1983,2191,3705,3610,2155, // 3040
252 3706,1912,1513,1614,6153,1988, 646, 392,2304,1589,3320,3039,1826,1239,1352,1340, // 3056
253 2916, 505,2567,1709,1437,2408,2547, 906,6154,2672, 384,1458,1594,1100,1329, 710, // 3072
254 423,3531,2064,2231,2622,1989,2673,1087,1882, 333, 841,3005,1296,2882,2379, 580, // 3088
255 1937,1827,1293,2585, 601, 574, 249,1772,4118,2079,1120, 645, 901,1176,1690, 795, // 3104
256 2207, 478,1434, 516,1190,1530, 761,2080, 930,1264, 355, 435,1552, 644,1791, 987, // 3120
257 220,1364,1163,1121,1538, 306,2169,1327,1222, 546,2645, 218, 241, 610,1704,3321, // 3136
258 1984,1839,1966,2528, 451,6155,2586,3707,2568, 907,3178, 254,2947, 186,1845,4650, // 3152
259 745, 432,1757, 428,1633, 888,2246,2221,2489,3611,2118,1258,1265, 956,3127,1784, // 3168
260 4337,2490, 319, 510, 119, 457,3612, 274,2035,2007,4651,1409,3128, 970,2758, 590, // 3184
261 2800, 661,2247,4652,2008,3950,1420,1549,3080,3322,3951,1651,1375,2111, 485,2491, // 3200
262 1429,1156,6156,2548,2183,1495, 831,1840,2529,2446, 501,1657, 307,1894,3247,1341, // 3216
263 666, 899,2156,1539,2549,1559, 886, 349,2208,3081,2305,1736,3824,2170,2759,1014, // 3232
264 1913,1386, 542,1397,2948, 490, 368, 716, 362, 159, 282,2569,1129,1658,1288,1750, // 3248
265 2674, 276, 649,2016, 751,1496, 658,1818,1284,1862,2209,2087,2512,3451, 622,2834, // 3264
266 376, 117,1060,2053,1208,1721,1101,1443, 247,1250,3179,1792,3952,2760,2398,3953, // 3280
267 6157,2144,3708, 446,2432,1151,2570,3452,2447,2761,2835,1210,2448,3082, 424,2222, // 3296
268 1251,2449,2119,2836, 504,1581,4338, 602, 817, 857,3825,2349,2306, 357,3826,1470, // 3312
269 1883,2883, 255, 958, 929,2917,3248, 302,4653,1050,1271,1751,2307,1952,1430,2697, // 3328
270 2719,2359, 354,3180, 777, 158,2036,4339,1659,4340,4654,2308,2949,2248,1146,2232, // 3344
271 3532,2720,1696,2623,3827,6158,3129,1550,2698,1485,1297,1428, 637, 931,2721,2145, // 3360
272 914,2550,2587, 81,2450, 612, 827,2646,1242,4655,1118,2884, 472,1855,3181,3533, // 3376
273 3534, 569,1353,2699,1244,1758,2588,4119,2009,2762,2171,3709,1312,1531,6159,1152, // 3392
274 1938, 134,1830, 471,3710,2276,1112,1535,3323,3453,3535, 982,1337,2950, 488, 826, // 3408
275 674,1058,1628,4120,2017, 522,2399, 211, 568,1367,3454, 350, 293,1872,1139,3249, // 3424
276 1399,1946,3006,1300,2360,3324, 588, 736,6160,2606, 744, 669,3536,3828,6161,1358, // 3440
277 199, 723, 848, 933, 851,1939,1505,1514,1338,1618,1831,4656,1634,3613, 443,2740, // 3456
278 3829, 717,1947, 491,1914,6162,2551,1542,4121,1025,6163,1099,1223, 198,3040,2722, // 3472
279 370, 410,1905,2589, 998,1248,3182,2380, 519,1449,4122,1710, 947, 928,1153,4341, // 3488
280 2277, 344,2624,1511, 615, 105, 161,1212,1076,1960,3130,2054,1926,1175,1906,2473, // 3504
281 414,1873,2801,6164,2309, 315,1319,3325, 318,2018,2146,2157, 963, 631, 223,4342, // 3520
282 4343,2675, 479,3711,1197,2625,3712,2676,2361,6165,4344,4123,6166,2451,3183,1886, // 3536
283 2184,1674,1330,1711,1635,1506, 799, 219,3250,3083,3954,1677,3713,3326,2081,3614, // 3552
284 1652,2073,4657,1147,3041,1752, 643,1961, 147,1974,3955,6167,1716,2037, 918,3007, // 3568
285 1994, 120,1537, 118, 609,3184,4345, 740,3455,1219, 332,1615,3830,6168,1621,2980, // 3584
286 1582, 783, 212, 553,2350,3714,1349,2433,2082,4124, 889,6169,2310,1275,1410, 973, // 3600
287 166,1320,3456,1797,1215,3185,2885,1846,2590,2763,4658, 629, 822,3008, 763, 940, // 3616
288 1990,2862, 439,2409,1566,1240,1622, 926,1282,1907,2764, 654,2210,1607, 327,1130, // 3632
289 3956,1678,1623,6170,2434,2192, 686, 608,3831,3715, 903,3957,3042,6171,2741,1522, // 3648
290 1915,1105,1555,2552,1359, 323,3251,4346,3457, 738,1354,2553,2311,2334,1828,2003, // 3664
291 3832,1753,2351,1227,6172,1887,4125,1478,6173,2410,1874,1712,1847, 520,1204,2607, // 3680
292 264,4659, 836,2677,2102, 600,4660,3833,2278,3084,6174,4347,3615,1342, 640, 532, // 3696
293 543,2608,1888,2400,2591,1009,4348,1497, 341,1737,3616,2723,1394, 529,3252,1321, // 3712
294 983,4661,1515,2120, 971,2592, 924, 287,1662,3186,4349,2700,4350,1519, 908,1948, // 3728
295 2452, 156, 796,1629,1486,2223,2055, 694,4126,1259,1036,3392,1213,2249,2742,1889, // 3744
296 1230,3958,1015, 910, 408, 559,3617,4662, 746, 725, 935,4663,3959,3009,1289, 563, // 3760
297 867,4664,3960,1567,2981,2038,2626, 988,2263,2381,4351, 143,2374, 704,1895,6175, // 3776
298 1188,3716,2088, 673,3085,2362,4352, 484,1608,1921,2765,2918, 215, 904,3618,3537, // 3792
299 894, 509, 976,3043,2701,3961,4353,2837,2982, 498,6176,6177,1102,3538,1332,3393, // 3808
300 1487,1636,1637, 233, 245,3962, 383, 650, 995,3044, 460,1520,1206,2352, 749,3327, // 3824
301 530, 700, 389,1438,1560,1773,3963,2264, 719,2951,2724,3834, 870,1832,1644,1000, // 3840
302 839,2474,3717, 197,1630,3394, 365,2886,3964,1285,2133, 734, 922, 818,1106, 732, // 3856
303 480,2083,1774,3458, 923,2279,1350, 221,3086, 85,2233,2234,3835,1585,3010,2147, // 3872
304 1387,1705,2382,1619,2475, 133, 239,2802,1991,1016,2084,2383, 411,2838,1113, 651, // 3888
305 1985,1160,3328, 990,1863,3087,1048,1276,2647, 265,2627,1599,3253,2056, 150, 638, // 3904
306 2019, 656, 853, 326,1479, 680,1439,4354,1001,1759, 413,3459,3395,2492,1431, 459, // 3920
307 4355,1125,3329,2265,1953,1450,2065,2863, 849, 351,2678,3131,3254,3255,1104,1577, // 3936
308 227,1351,1645,2453,2193,1421,2887, 812,2121, 634, 95,2435, 201,2312,4665,1646, // 3952
309 1671,2743,1601,2554,2702,2648,2280,1315,1366,2089,3132,1573,3718,3965,1729,1189, // 3968
310 328,2679,1077,1940,1136, 558,1283, 964,1195, 621,2074,1199,1743,3460,3619,1896, // 3984
311 1916,1890,3836,2952,1154,2112,1064, 862, 378,3011,2066,2113,2803,1568,2839,6178, // 4000
312 3088,2919,1941,1660,2004,1992,2194, 142, 707,1590,1708,1624,1922,1023,1836,1233, // 4016
313 1004,2313, 789, 741,3620,6179,1609,2411,1200,4127,3719,3720,4666,2057,3721, 593, // 4032
314 2840, 367,2920,1878,6180,3461,1521, 628,1168, 692,2211,2649, 300, 720,2067,2571, // 4048
315 2953,3396, 959,2504,3966,3539,3462,1977, 701,6181, 954,1043, 800, 681, 183,3722, // 4064
316 1803,1730,3540,4128,2103, 815,2314, 174, 467, 230,2454,1093,2134, 755,3541,3397, // 4080
317 1141,1162,6182,1738,2039, 270,3256,2513,1005,1647,2185,3837, 858,1679,1897,1719, // 4096
318 2954,2324,1806, 402, 670, 167,4129,1498,2158,2104, 750,6183, 915, 189,1680,1551, // 4112
319 455,4356,1501,2455, 405,1095,2955, 338,1586,1266,1819, 570, 641,1324, 237,1556, // 4128
320 2650,1388,3723,6184,1368,2384,1343,1978,3089,2436, 879,3724, 792,1191, 758,3012, // 4144
321 1411,2135,1322,4357, 240,4667,1848,3725,1574,6185, 420,3045,1546,1391, 714,4358, // 4160
322 1967, 941,1864, 863, 664, 426, 560,1731,2680,1785,2864,1949,2363, 403,3330,1415, // 4176
323 1279,2136,1697,2335, 204, 721,2097,3838, 90,6186,2085,2505, 191,3967, 124,2148, // 4192
324 1376,1798,1178,1107,1898,1405, 860,4359,1243,1272,2375,2983,1558,2456,1638, 113, // 4208
325 3621, 578,1923,2609, 880, 386,4130, 784,2186,2266,1422,2956,2172,1722, 497, 263, // 4224
326 2514,1267,2412,2610, 177,2703,3542, 774,1927,1344, 616,1432,1595,1018, 172,4360, // 4240
327 2325, 911,4361, 438,1468,3622, 794,3968,2024,2173,1681,1829,2957, 945, 895,3090, // 4256
328 575,2212,2476, 475,2401,2681, 785,2744,1745,2293,2555,1975,3133,2865, 394,4668, // 4272
329 3839, 635,4131, 639, 202,1507,2195,2766,1345,1435,2572,3726,1908,1184,1181,2457, // 4288
330 3727,3134,4362, 843,2611, 437, 916,4669, 234, 769,1884,3046,3047,3623, 833,6187, // 4304
331 1639,2250,2402,1355,1185,2010,2047, 999, 525,1732,1290,1488,2612, 948,1578,3728, // 4320
332 2413,2477,1216,2725,2159, 334,3840,1328,3624,2921,1525,4132, 564,1056, 891,4363, // 4336
333 1444,1698,2385,2251,3729,1365,2281,2235,1717,6188, 864,3841,2515, 444, 527,2767, // 4352
334 2922,3625, 544, 461,6189, 566, 209,2437,3398,2098,1065,2068,3331,3626,3257,2137, // 4368 //last 512
335
336 /***************************************************************************************
337 *Everything below is of no interest for detection purpose *
338 ***************************************************************************************
339
340 2138,2122,3730,2888,1995,1820,1044,6190,6191,6192,6193,6194,6195,6196,6197,6198, // 4384
341 6199,6200,6201,6202,6203,6204,6205,4670,6206,6207,6208,6209,6210,6211,6212,6213, // 4400
342 6214,6215,6216,6217,6218,6219,6220,6221,6222,6223,6224,6225,6226,6227,6228,6229, // 4416
343 6230,6231,6232,6233,6234,6235,6236,6237,3187,6238,6239,3969,6240,6241,6242,6243, // 4432
344 6244,4671,6245,6246,4672,6247,6248,4133,6249,6250,4364,6251,2923,2556,2613,4673, // 4448
345 4365,3970,6252,6253,6254,6255,4674,6256,6257,6258,2768,2353,4366,4675,4676,3188, // 4464
346 4367,3463,6259,4134,4677,4678,6260,2267,6261,3842,3332,4368,3543,6262,6263,6264, // 4480
347 3013,1954,1928,4135,4679,6265,6266,2478,3091,6267,4680,4369,6268,6269,1699,6270, // 4496
348 3544,4136,4681,6271,4137,6272,4370,2804,6273,6274,2593,3971,3972,4682,6275,2236, // 4512
349 4683,6276,6277,4684,6278,6279,4138,3973,4685,6280,6281,3258,6282,6283,6284,6285, // 4528
350 3974,4686,2841,3975,6286,6287,3545,6288,6289,4139,4687,4140,6290,4141,6291,4142, // 4544
351 6292,6293,3333,6294,6295,6296,4371,6297,3399,6298,6299,4372,3976,6300,6301,6302, // 4560
352 4373,6303,6304,3843,3731,6305,4688,4374,6306,6307,3259,2294,6308,3732,2530,4143, // 4576
353 6309,4689,6310,6311,6312,3048,6313,6314,4690,3733,2237,6315,6316,2282,3334,6317, // 4592
354 6318,3844,6319,6320,4691,6321,3400,4692,6322,4693,6323,3049,6324,4375,6325,3977, // 4608
355 6326,6327,6328,3546,6329,4694,3335,6330,4695,4696,6331,6332,6333,6334,4376,3978, // 4624
356 6335,4697,3979,4144,6336,3980,4698,6337,6338,6339,6340,6341,4699,4700,4701,6342, // 4640
357 6343,4702,6344,6345,4703,6346,6347,4704,6348,4705,4706,3135,6349,4707,6350,4708, // 4656
358 6351,4377,6352,4709,3734,4145,6353,2506,4710,3189,6354,3050,4711,3981,6355,3547, // 4672
359 3014,4146,4378,3735,2651,3845,3260,3136,2224,1986,6356,3401,6357,4712,2594,3627, // 4688
360 3137,2573,3736,3982,4713,3628,4714,4715,2682,3629,4716,6358,3630,4379,3631,6359, // 4704
361 6360,6361,3983,6362,6363,6364,6365,4147,3846,4717,6366,6367,3737,2842,6368,4718, // 4720
362 2628,6369,3261,6370,2386,6371,6372,3738,3984,4719,3464,4720,3402,6373,2924,3336, // 4736
363 4148,2866,6374,2805,3262,4380,2704,2069,2531,3138,2806,2984,6375,2769,6376,4721, // 4752
364 4722,3403,6377,6378,3548,6379,6380,2705,3092,1979,4149,2629,3337,2889,6381,3338, // 4768
365 4150,2557,3339,4381,6382,3190,3263,3739,6383,4151,4723,4152,2558,2574,3404,3191, // 4784
366 6384,6385,4153,6386,4724,4382,6387,6388,4383,6389,6390,4154,6391,4725,3985,6392, // 4800
367 3847,4155,6393,6394,6395,6396,6397,3465,6398,4384,6399,6400,6401,6402,6403,6404, // 4816
368 4156,6405,6406,6407,6408,2123,6409,6410,2326,3192,4726,6411,6412,6413,6414,4385, // 4832
369 4157,6415,6416,4158,6417,3093,3848,6418,3986,6419,6420,3849,6421,6422,6423,4159, // 4848
370 6424,6425,4160,6426,3740,6427,6428,6429,6430,3987,6431,4727,6432,2238,6433,6434, // 4864
371 4386,3988,6435,6436,3632,6437,6438,2843,6439,6440,6441,6442,3633,6443,2958,6444, // 4880
372 6445,3466,6446,2364,4387,3850,6447,4388,2959,3340,6448,3851,6449,4728,6450,6451, // 4896
373 3264,4729,6452,3193,6453,4389,4390,2706,3341,4730,6454,3139,6455,3194,6456,3051, // 4912
374 2124,3852,1602,4391,4161,3853,1158,3854,4162,3989,4392,3990,4731,4732,4393,2040, // 4928
375 4163,4394,3265,6457,2807,3467,3855,6458,6459,6460,3991,3468,4733,4734,6461,3140, // 4944
376 2960,6462,4735,6463,6464,6465,6466,4736,4737,4738,4739,6467,6468,4164,2403,3856, // 4960
377 6469,6470,2770,2844,6471,4740,6472,6473,6474,6475,6476,6477,6478,3195,6479,4741, // 4976
378 4395,6480,2867,6481,4742,2808,6482,2493,4165,6483,6484,6485,6486,2295,4743,6487, // 4992
379 6488,6489,3634,6490,6491,6492,6493,6494,6495,6496,2985,4744,6497,6498,4745,6499, // 5008
380 6500,2925,3141,4166,6501,6502,4746,6503,6504,4747,6505,6506,6507,2890,6508,6509, // 5024
381 6510,6511,6512,6513,6514,6515,6516,6517,6518,6519,3469,4167,6520,6521,6522,4748, // 5040
382 4396,3741,4397,4749,4398,3342,2125,4750,6523,4751,4752,4753,3052,6524,2961,4168, // 5056
383 6525,4754,6526,4755,4399,2926,4169,6527,3857,6528,4400,4170,6529,4171,6530,6531, // 5072
384 2595,6532,6533,6534,6535,3635,6536,6537,6538,6539,6540,6541,6542,4756,6543,6544, // 5088
385 6545,6546,6547,6548,4401,6549,6550,6551,6552,4402,3405,4757,4403,6553,6554,6555, // 5104
386 4172,3742,6556,6557,6558,3992,3636,6559,6560,3053,2726,6561,3549,4173,3054,4404, // 5120
387 6562,6563,3993,4405,3266,3550,2809,4406,6564,6565,6566,4758,4759,6567,3743,6568, // 5136
388 4760,3744,4761,3470,6569,6570,6571,4407,6572,3745,4174,6573,4175,2810,4176,3196, // 5152
389 4762,6574,4177,6575,6576,2494,2891,3551,6577,6578,3471,6579,4408,6580,3015,3197, // 5168
390 6581,3343,2532,3994,3858,6582,3094,3406,4409,6583,2892,4178,4763,4410,3016,4411, // 5184
391 6584,3995,3142,3017,2683,6585,4179,6586,6587,4764,4412,6588,6589,4413,6590,2986, // 5200
392 6591,2962,3552,6592,2963,3472,6593,6594,4180,4765,6595,6596,2225,3267,4414,6597, // 5216
393 3407,3637,4766,6598,6599,3198,6600,4415,6601,3859,3199,6602,3473,4767,2811,4416, // 5232
394 1856,3268,3200,2575,3996,3997,3201,4417,6603,3095,2927,6604,3143,6605,2268,6606, // 5248
395 3998,3860,3096,2771,6607,6608,3638,2495,4768,6609,3861,6610,3269,2745,4769,4181, // 5264
396 3553,6611,2845,3270,6612,6613,6614,3862,6615,6616,4770,4771,6617,3474,3999,4418, // 5280
397 4419,6618,3639,3344,6619,4772,4182,6620,2126,6621,6622,6623,4420,4773,6624,3018, // 5296
398 6625,4774,3554,6626,4183,2025,3746,6627,4184,2707,6628,4421,4422,3097,1775,4185, // 5312
399 3555,6629,6630,2868,6631,6632,4423,6633,6634,4424,2414,2533,2928,6635,4186,2387, // 5328
400 6636,4775,6637,4187,6638,1891,4425,3202,3203,6639,6640,4776,6641,3345,6642,6643, // 5344
401 3640,6644,3475,3346,3641,4000,6645,3144,6646,3098,2812,4188,3642,3204,6647,3863, // 5360
402 3476,6648,3864,6649,4426,4001,6650,6651,6652,2576,6653,4189,4777,6654,6655,6656, // 5376
403 2846,6657,3477,3205,4002,6658,4003,6659,3347,2252,6660,6661,6662,4778,6663,6664, // 5392
404 6665,6666,6667,6668,6669,4779,4780,2048,6670,3478,3099,6671,3556,3747,4004,6672, // 5408
405 6673,6674,3145,4005,3748,6675,6676,6677,6678,6679,3408,6680,6681,6682,6683,3206, // 5424
406 3207,6684,6685,4781,4427,6686,4782,4783,4784,6687,6688,6689,4190,6690,6691,3479, // 5440
407 6692,2746,6693,4428,6694,6695,6696,6697,6698,6699,4785,6700,6701,3208,2727,6702, // 5456
408 3146,6703,6704,3409,2196,6705,4429,6706,6707,6708,2534,1996,6709,6710,6711,2747, // 5472
409 6712,6713,6714,4786,3643,6715,4430,4431,6716,3557,6717,4432,4433,6718,6719,6720, // 5488
410 6721,3749,6722,4006,4787,6723,6724,3644,4788,4434,6725,6726,4789,2772,6727,6728, // 5504
411 6729,6730,6731,2708,3865,2813,4435,6732,6733,4790,4791,3480,6734,6735,6736,6737, // 5520
412 4436,3348,6738,3410,4007,6739,6740,4008,6741,6742,4792,3411,4191,6743,6744,6745, // 5536
413 6746,6747,3866,6748,3750,6749,6750,6751,6752,6753,6754,6755,3867,6756,4009,6757, // 5552
414 4793,4794,6758,2814,2987,6759,6760,6761,4437,6762,6763,6764,6765,3645,6766,6767, // 5568
415 3481,4192,6768,3751,6769,6770,2174,6771,3868,3752,6772,6773,6774,4193,4795,4438, // 5584
416 3558,4796,4439,6775,4797,6776,6777,4798,6778,4799,3559,4800,6779,6780,6781,3482, // 5600
417 6782,2893,6783,6784,4194,4801,4010,6785,6786,4440,6787,4011,6788,6789,6790,6791, // 5616
418 6792,6793,4802,6794,6795,6796,4012,6797,6798,6799,6800,3349,4803,3483,6801,4804, // 5632
419 4195,6802,4013,6803,6804,4196,6805,4014,4015,6806,2847,3271,2848,6807,3484,6808, // 5648
420 6809,6810,4441,6811,4442,4197,4443,3272,4805,6812,3412,4016,1579,6813,6814,4017, // 5664
421 6815,3869,6816,2964,6817,4806,6818,6819,4018,3646,6820,6821,4807,4019,4020,6822, // 5680
422 6823,3560,6824,6825,4021,4444,6826,4198,6827,6828,4445,6829,6830,4199,4808,6831, // 5696
423 6832,6833,3870,3019,2458,6834,3753,3413,3350,6835,4809,3871,4810,3561,4446,6836, // 5712
424 6837,4447,4811,4812,6838,2459,4448,6839,4449,6840,6841,4022,3872,6842,4813,4814, // 5728
425 6843,6844,4815,4200,4201,4202,6845,4023,6846,6847,4450,3562,3873,6848,6849,4816, // 5744
426 4817,6850,4451,4818,2139,6851,3563,6852,6853,3351,6854,6855,3352,4024,2709,3414, // 5760
427 4203,4452,6856,4204,6857,6858,3874,3875,6859,6860,4819,6861,6862,6863,6864,4453, // 5776
428 3647,6865,6866,4820,6867,6868,6869,6870,4454,6871,2869,6872,6873,4821,6874,3754, // 5792
429 6875,4822,4205,6876,6877,6878,3648,4206,4455,6879,4823,6880,4824,3876,6881,3055, // 5808
430 4207,6882,3415,6883,6884,6885,4208,4209,6886,4210,3353,6887,3354,3564,3209,3485, // 5824
431 2652,6888,2728,6889,3210,3755,6890,4025,4456,6891,4825,6892,6893,6894,6895,4211, // 5840
432 6896,6897,6898,4826,6899,6900,4212,6901,4827,6902,2773,3565,6903,4828,6904,6905, // 5856
433 6906,6907,3649,3650,6908,2849,3566,6909,3567,3100,6910,6911,6912,6913,6914,6915, // 5872
434 4026,6916,3355,4829,3056,4457,3756,6917,3651,6918,4213,3652,2870,6919,4458,6920, // 5888
435 2438,6921,6922,3757,2774,4830,6923,3356,4831,4832,6924,4833,4459,3653,2507,6925, // 5904
436 4834,2535,6926,6927,3273,4027,3147,6928,3568,6929,6930,6931,4460,6932,3877,4461, // 5920
437 2729,3654,6933,6934,6935,6936,2175,4835,2630,4214,4028,4462,4836,4215,6937,3148, // 5936
438 4216,4463,4837,4838,4217,6938,6939,2850,4839,6940,4464,6941,6942,6943,4840,6944, // 5952
439 4218,3274,4465,6945,6946,2710,6947,4841,4466,6948,6949,2894,6950,6951,4842,6952, // 5968
440 4219,3057,2871,6953,6954,6955,6956,4467,6957,2711,6958,6959,6960,3275,3101,4843, // 5984
441 6961,3357,3569,6962,4844,6963,6964,4468,4845,3570,6965,3102,4846,3758,6966,4847, // 6000
442 3878,4848,4849,4029,6967,2929,3879,4850,4851,6968,6969,1733,6970,4220,6971,6972, // 6016
443 6973,6974,6975,6976,4852,6977,6978,6979,6980,6981,6982,3759,6983,6984,6985,3486, // 6032
444 3487,6986,3488,3416,6987,6988,6989,6990,6991,6992,6993,6994,6995,6996,6997,4853, // 6048
445 6998,6999,4030,7000,7001,3211,7002,7003,4221,7004,7005,3571,4031,7006,3572,7007, // 6064
446 2614,4854,2577,7008,7009,2965,3655,3656,4855,2775,3489,3880,4222,4856,3881,4032, // 6080
447 3882,3657,2730,3490,4857,7010,3149,7011,4469,4858,2496,3491,4859,2283,7012,7013, // 6096
448 7014,2365,4860,4470,7015,7016,3760,7017,7018,4223,1917,7019,7020,7021,4471,7022, // 6112
449 2776,4472,7023,7024,7025,7026,4033,7027,3573,4224,4861,4034,4862,7028,7029,1929, // 6128
450 3883,4035,7030,4473,3058,7031,2536,3761,3884,7032,4036,7033,2966,2895,1968,4474, // 6144
451 3276,4225,3417,3492,4226,2105,7034,7035,1754,2596,3762,4227,4863,4475,3763,4864, // 6160
452 3764,2615,2777,3103,3765,3658,3418,4865,2296,3766,2815,7036,7037,7038,3574,2872, // 6176
453 3277,4476,7039,4037,4477,7040,7041,4038,7042,7043,7044,7045,7046,7047,2537,7048, // 6192
454 7049,7050,7051,7052,7053,7054,4478,7055,7056,3767,3659,4228,3575,7057,7058,4229, // 6208
455 7059,7060,7061,3660,7062,3212,7063,3885,4039,2460,7064,7065,7066,7067,7068,7069, // 6224
456 7070,7071,7072,7073,7074,4866,3768,4867,7075,7076,7077,7078,4868,3358,3278,2653, // 6240
457 7079,7080,4479,3886,7081,7082,4869,7083,7084,7085,7086,7087,7088,2538,7089,7090, // 6256
458 7091,4040,3150,3769,4870,4041,2896,3359,4230,2930,7092,3279,7093,2967,4480,3213, // 6272
459 4481,3661,7094,7095,7096,7097,7098,7099,7100,7101,7102,2461,3770,7103,7104,4231, // 6288
460 3151,7105,7106,7107,4042,3662,7108,7109,4871,3663,4872,4043,3059,7110,7111,7112, // 6304
461 3493,2988,7113,4873,7114,7115,7116,3771,4874,7117,7118,4232,4875,7119,3576,2336, // 6320
462 4876,7120,4233,3419,4044,4877,4878,4482,4483,4879,4484,4234,7121,3772,4880,1045, // 6336
463 3280,3664,4881,4882,7122,7123,7124,7125,4883,7126,2778,7127,4485,4486,7128,4884, // 6352
464 3214,3887,7129,7130,3215,7131,4885,4045,7132,7133,4046,7134,7135,7136,7137,7138, // 6368
465 7139,7140,7141,7142,7143,4235,7144,4886,7145,7146,7147,4887,7148,7149,7150,4487, // 6384
466 4047,4488,7151,7152,4888,4048,2989,3888,7153,3665,7154,4049,7155,7156,7157,7158, // 6400
467 7159,7160,2931,4889,4890,4489,7161,2631,3889,4236,2779,7162,7163,4891,7164,3060, // 6416
468 7165,1672,4892,7166,4893,4237,3281,4894,7167,7168,3666,7169,3494,7170,7171,4050, // 6432
469 7172,7173,3104,3360,3420,4490,4051,2684,4052,7174,4053,7175,7176,7177,2253,4054, // 6448
470 7178,7179,4895,7180,3152,3890,3153,4491,3216,7181,7182,7183,2968,4238,4492,4055, // 6464
471 7184,2990,7185,2479,7186,7187,4493,7188,7189,7190,7191,7192,4896,7193,4897,2969, // 6480
472 4494,4898,7194,3495,7195,7196,4899,4495,7197,3105,2731,7198,4900,7199,7200,7201, // 6496
473 4056,7202,3361,7203,7204,4496,4901,4902,7205,4497,7206,7207,2315,4903,7208,4904, // 6512
474 7209,4905,2851,7210,7211,3577,7212,3578,4906,7213,4057,3667,4907,7214,4058,2354, // 6528
475 3891,2376,3217,3773,7215,7216,7217,7218,7219,4498,7220,4908,3282,2685,7221,3496, // 6544
476 4909,2632,3154,4910,7222,2337,7223,4911,7224,7225,7226,4912,4913,3283,4239,4499, // 6560
477 7227,2816,7228,7229,7230,7231,7232,7233,7234,4914,4500,4501,7235,7236,7237,2686, // 6576
478 7238,4915,7239,2897,4502,7240,4503,7241,2516,7242,4504,3362,3218,7243,7244,7245, // 6592
479 4916,7246,7247,4505,3363,7248,7249,7250,7251,3774,4506,7252,7253,4917,7254,7255, // 6608
480 3284,2991,4918,4919,3219,3892,4920,3106,3497,4921,7256,7257,7258,4922,7259,4923, // 6624
481 3364,4507,4508,4059,7260,4240,3498,7261,7262,4924,7263,2992,3893,4060,3220,7264, // 6640
482 7265,7266,7267,7268,7269,4509,3775,7270,2817,7271,4061,4925,4510,3776,7272,4241, // 6656
483 4511,3285,7273,7274,3499,7275,7276,7277,4062,4512,4926,7278,3107,3894,7279,7280, // 6672
484 4927,7281,4513,7282,7283,3668,7284,7285,4242,4514,4243,7286,2058,4515,4928,4929, // 6688
485 4516,7287,3286,4244,7288,4517,7289,7290,7291,3669,7292,7293,4930,4931,4932,2355, // 6704
486 4933,7294,2633,4518,7295,4245,7296,7297,4519,7298,7299,4520,4521,4934,7300,4246, // 6720
487 4522,7301,7302,7303,3579,7304,4247,4935,7305,4936,7306,7307,7308,7309,3777,7310, // 6736
488 4523,7311,7312,7313,4248,3580,7314,4524,3778,4249,7315,3581,7316,3287,7317,3221, // 6752
489 7318,4937,7319,7320,7321,7322,7323,7324,4938,4939,7325,4525,7326,7327,7328,4063, // 6768
490 7329,7330,4940,7331,7332,4941,7333,4526,7334,3500,2780,1741,4942,2026,1742,7335, // 6784
491 7336,3582,4527,2388,7337,7338,7339,4528,7340,4250,4943,7341,7342,7343,4944,7344, // 6800
492 7345,7346,3020,7347,4945,7348,7349,7350,7351,3895,7352,3896,4064,3897,7353,7354, // 6816
493 7355,4251,7356,7357,3898,7358,3779,7359,3780,3288,7360,7361,4529,7362,4946,4530, // 6832
494 2027,7363,3899,4531,4947,3222,3583,7364,4948,7365,7366,7367,7368,4949,3501,4950, // 6848
495 3781,4951,4532,7369,2517,4952,4252,4953,3155,7370,4954,4955,4253,2518,4533,7371, // 6864
496 7372,2712,4254,7373,7374,7375,3670,4956,3671,7376,2389,3502,4065,7377,2338,7378, // 6880
497 7379,7380,7381,3061,7382,4957,7383,7384,7385,7386,4958,4534,7387,7388,2993,7389, // 6896
498 3062,7390,4959,7391,7392,7393,4960,3108,4961,7394,4535,7395,4962,3421,4536,7396, // 6912
499 4963,7397,4964,1857,7398,4965,7399,7400,2176,3584,4966,7401,7402,3422,4537,3900, // 6928
500 3585,7403,3782,7404,2852,7405,7406,7407,4538,3783,2654,3423,4967,4539,7408,3784, // 6944
501 3586,2853,4540,4541,7409,3901,7410,3902,7411,7412,3785,3109,2327,3903,7413,7414, // 6960
502 2970,4066,2932,7415,7416,7417,3904,3672,3424,7418,4542,4543,4544,7419,4968,7420, // 6976
503 7421,4255,7422,7423,7424,7425,7426,4067,7427,3673,3365,4545,7428,3110,2559,3674, // 6992
504 7429,7430,3156,7431,7432,3503,7433,3425,4546,7434,3063,2873,7435,3223,4969,4547, // 7008
505 4548,2898,4256,4068,7436,4069,3587,3786,2933,3787,4257,4970,4971,3788,7437,4972, // 7024
506 3064,7438,4549,7439,7440,7441,7442,7443,4973,3905,7444,2874,7445,7446,7447,7448, // 7040
507 3021,7449,4550,3906,3588,4974,7450,7451,3789,3675,7452,2578,7453,4070,7454,7455, // 7056
508 7456,4258,3676,7457,4975,7458,4976,4259,3790,3504,2634,4977,3677,4551,4260,7459, // 7072
509 7460,7461,7462,3907,4261,4978,7463,7464,7465,7466,4979,4980,7467,7468,2213,4262, // 7088
510 7469,7470,7471,3678,4981,7472,2439,7473,4263,3224,3289,7474,3908,2415,4982,7475, // 7104
511 4264,7476,4983,2655,7477,7478,2732,4552,2854,2875,7479,7480,4265,7481,4553,4984, // 7120
512 7482,7483,4266,7484,3679,3366,3680,2818,2781,2782,3367,3589,4554,3065,7485,4071, // 7136
513 2899,7486,7487,3157,2462,4072,4555,4073,4985,4986,3111,4267,2687,3368,4556,4074, // 7152
514 3791,4268,7488,3909,2783,7489,2656,1962,3158,4557,4987,1963,3159,3160,7490,3112, // 7168
515 4988,4989,3022,4990,4991,3792,2855,7491,7492,2971,4558,7493,7494,4992,7495,7496, // 7184
516 7497,7498,4993,7499,3426,4559,4994,7500,3681,4560,4269,4270,3910,7501,4075,4995, // 7200
517 4271,7502,7503,4076,7504,4996,7505,3225,4997,4272,4077,2819,3023,7506,7507,2733, // 7216
518 4561,7508,4562,7509,3369,3793,7510,3590,2508,7511,7512,4273,3113,2994,2616,7513, // 7232
519 7514,7515,7516,7517,7518,2820,3911,4078,2748,7519,7520,4563,4998,7521,7522,7523, // 7248
520 7524,4999,4274,7525,4564,3682,2239,4079,4565,7526,7527,7528,7529,5000,7530,7531, // 7264
521 5001,4275,3794,7532,7533,7534,3066,5002,4566,3161,7535,7536,4080,7537,3162,7538, // 7280
522 7539,4567,7540,7541,7542,7543,7544,7545,5003,7546,4568,7547,7548,7549,7550,7551, // 7296
523 7552,7553,7554,7555,7556,5004,7557,7558,7559,5005,7560,3795,7561,4569,7562,7563, // 7312
524 7564,2821,3796,4276,4277,4081,7565,2876,7566,5006,7567,7568,2900,7569,3797,3912, // 7328
525 7570,7571,7572,4278,7573,7574,7575,5007,7576,7577,5008,7578,7579,4279,2934,7580, // 7344
526 7581,5009,7582,4570,7583,4280,7584,7585,7586,4571,4572,3913,7587,4573,3505,7588, // 7360
527 5010,7589,7590,7591,7592,3798,4574,7593,7594,5011,7595,4281,7596,7597,7598,4282, // 7376
528 5012,7599,7600,5013,3163,7601,5014,7602,3914,7603,7604,2734,4575,4576,4577,7605, // 7392
529 7606,7607,7608,7609,3506,5015,4578,7610,4082,7611,2822,2901,2579,3683,3024,4579, // 7408
530 3507,7612,4580,7613,3226,3799,5016,7614,7615,7616,7617,7618,7619,7620,2995,3290, // 7424
531 7621,4083,7622,5017,7623,7624,7625,7626,7627,4581,3915,7628,3291,7629,5018,7630, // 7440
532 7631,7632,7633,4084,7634,7635,3427,3800,7636,7637,4582,7638,5019,4583,5020,7639, // 7456
533 3916,7640,3801,5021,4584,4283,7641,7642,3428,3591,2269,7643,2617,7644,4585,3592, // 7472
534 7645,4586,2902,7646,7647,3227,5022,7648,4587,7649,4284,7650,7651,7652,4588,2284, // 7488
535 7653,5023,7654,7655,7656,4589,5024,3802,7657,7658,5025,3508,4590,7659,7660,7661, // 7504
536 1969,5026,7662,7663,3684,1821,2688,7664,2028,2509,4285,7665,2823,1841,7666,2689, // 7520
537 3114,7667,3917,4085,2160,5027,5028,2972,7668,5029,7669,7670,7671,3593,4086,7672, // 7536
538 4591,4087,5030,3803,7673,7674,7675,7676,7677,7678,7679,4286,2366,4592,4593,3067, // 7552
539 2328,7680,7681,4594,3594,3918,2029,4287,7682,5031,3919,3370,4288,4595,2856,7683, // 7568
540 3509,7684,7685,5032,5033,7686,7687,3804,2784,7688,7689,7690,7691,3371,7692,7693, // 7584
541 2877,5034,7694,7695,3920,4289,4088,7696,7697,7698,5035,7699,5036,4290,5037,5038, // 7600
542 5039,7700,7701,7702,5040,5041,3228,7703,1760,7704,5042,3229,4596,2106,4089,7705, // 7616
543 4597,2824,5043,2107,3372,7706,4291,4090,5044,7707,4091,7708,5045,3025,3805,4598, // 7632
544 4292,4293,4294,3373,7709,4599,7710,5046,7711,7712,5047,5048,3806,7713,7714,7715, // 7648
545 5049,7716,7717,7718,7719,4600,5050,7720,7721,7722,5051,7723,4295,3429,7724,7725, // 7664
546 7726,7727,3921,7728,3292,5052,4092,7729,7730,7731,7732,7733,7734,7735,5053,5054, // 7680
547 7736,7737,7738,7739,3922,3685,7740,7741,7742,7743,2635,5055,7744,5056,4601,7745, // 7696
548 7746,2560,7747,7748,7749,7750,3923,7751,7752,7753,7754,7755,4296,2903,7756,7757, // 7712
549 7758,7759,7760,3924,7761,5057,4297,7762,7763,5058,4298,7764,4093,7765,7766,5059, // 7728
550 3925,7767,7768,7769,7770,7771,7772,7773,7774,7775,7776,3595,7777,4299,5060,4094, // 7744
551 7778,3293,5061,7779,7780,4300,7781,7782,4602,7783,3596,7784,7785,3430,2367,7786, // 7760
552 3164,5062,5063,4301,7787,7788,4095,5064,5065,7789,3374,3115,7790,7791,7792,7793, // 7776
553 7794,7795,7796,3597,4603,7797,7798,3686,3116,3807,5066,7799,7800,5067,7801,7802, // 7792
554 4604,4302,5068,4303,4096,7803,7804,3294,7805,7806,5069,4605,2690,7807,3026,7808, // 7808
555 7809,7810,7811,7812,7813,7814,7815,7816,7817,7818,7819,7820,7821,7822,7823,7824, // 7824
556 7825,7826,7827,7828,7829,7830,7831,7832,7833,7834,7835,7836,7837,7838,7839,7840, // 7840
557 7841,7842,7843,7844,7845,7846,7847,7848,7849,7850,7851,7852,7853,7854,7855,7856, // 7856
558 7857,7858,7859,7860,7861,7862,7863,7864,7865,7866,7867,7868,7869,7870,7871,7872, // 7872
559 7873,7874,7875,7876,7877,7878,7879,7880,7881,7882,7883,7884,7885,7886,7887,7888, // 7888
560 7889,7890,7891,7892,7893,7894,7895,7896,7897,7898,7899,7900,7901,7902,7903,7904, // 7904
561 7905,7906,7907,7908,7909,7910,7911,7912,7913,7914,7915,7916,7917,7918,7919,7920, // 7920
562 7921,7922,7923,7924,3926,7925,7926,7927,7928,7929,7930,7931,7932,7933,7934,7935, // 7936
563 7936,7937,7938,7939,7940,7941,7942,7943,7944,7945,7946,7947,7948,7949,7950,7951, // 7952
564 7952,7953,7954,7955,7956,7957,7958,7959,7960,7961,7962,7963,7964,7965,7966,7967, // 7968
565 7968,7969,7970,7971,7972,7973,7974,7975,7976,7977,7978,7979,7980,7981,7982,7983, // 7984
566 7984,7985,7986,7987,7988,7989,7990,7991,7992,7993,7994,7995,7996,7997,7998,7999, // 8000
567 8000,8001,8002,8003,8004,8005,8006,8007,8008,8009,8010,8011,8012,8013,8014,8015, // 8016
568 8016,8017,8018,8019,8020,8021,8022,8023,8024,8025,8026,8027,8028,8029,8030,8031, // 8032
569 8032,8033,8034,8035,8036,8037,8038,8039,8040,8041,8042,8043,8044,8045,8046,8047, // 8048
570 8048,8049,8050,8051,8052,8053,8054,8055,8056,8057,8058,8059,8060,8061,8062,8063, // 8064
571 8064,8065,8066,8067,8068,8069,8070,8071,8072,8073,8074,8075,8076,8077,8078,8079, // 8080
572 8080,8081,8082,8083,8084,8085,8086,8087,8088,8089,8090,8091,8092,8093,8094,8095, // 8096
573 8096,8097,8098,8099,8100,8101,8102,8103,8104,8105,8106,8107,8108,8109,8110,8111, // 8112
574 8112,8113,8114,8115,8116,8117,8118,8119,8120,8121,8122,8123,8124,8125,8126,8127, // 8128
575 8128,8129,8130,8131,8132,8133,8134,8135,8136,8137,8138,8139,8140,8141,8142,8143, // 8144
576 8144,8145,8146,8147,8148,8149,8150,8151,8152,8153,8154,8155,8156,8157,8158,8159, // 8160
577 8160,8161,8162,8163,8164,8165,8166,8167,8168,8169,8170,8171,8172,8173,8174,8175, // 8176
578 8176,8177,8178,8179,8180,8181,8182,8183,8184,8185,8186,8187,8188,8189,8190,8191, // 8192
579 8192,8193,8194,8195,8196,8197,8198,8199,8200,8201,8202,8203,8204,8205,8206,8207, // 8208
580 8208,8209,8210,8211,8212,8213,8214,8215,8216,8217,8218,8219,8220,8221,8222,8223, // 8224
581 8224,8225,8226,8227,8228,8229,8230,8231,8232,8233,8234,8235,8236,8237,8238,8239, // 8240
582 8240,8241,8242,8243,8244,8245,8246,8247,8248,8249,8250,8251,8252,8253,8254,8255, // 8256
583 8256,8257,8258,8259,8260,8261,8262,8263,8264,8265,8266,8267,8268,8269,8270,8271, // 8272
584 ****************************************************************************************/
585 };
586 }
+0
-82
src/org/mozilla/universalchardet/prober/distributionanalysis/SJISDistributionAnalysis.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.distributionanalysis;
38
39 public class SJISDistributionAnalysis extends JISDistributionAnalysis
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final int HIGHBYTE_BEGIN_1 = 0x81;
45 public static final int HIGHBYTE_END_1 = 0x9F;
46 public static final int HIGHBYTE_BEGIN_2 = 0xE0;
47 public static final int HIGHBYTE_END_2 = 0xEF;
48 public static final int LOWBYTE_BEGIN_1 = 0x40;
49 public static final int LOWBYTE_BEGIN_2 = 0x80;
50
51
52 ////////////////////////////////////////////////////////////////
53 // methods
54 ////////////////////////////////////////////////////////////////
55 public SJISDistributionAnalysis()
56 {
57 super();
58 }
59
60 @Override
61 protected int getOrder(final byte[] buf, int offset)
62 {
63 int order = -1;
64
65 int highbyte = buf[offset] & 0xFF;
66 if (highbyte >= HIGHBYTE_BEGIN_1 && highbyte <= HIGHBYTE_END_1) {
67 order = 188 * (highbyte - HIGHBYTE_BEGIN_1);
68 } else if (highbyte >= HIGHBYTE_BEGIN_2 && highbyte <= HIGHBYTE_END_2) {
69 order = 188 * (highbyte - HIGHBYTE_BEGIN_2 + 31);
70 } else {
71 return -1;
72 }
73 int lowbyte = buf[offset+1] & 0xFF;
74 order += lowbyte - LOWBYTE_BEGIN_1;
75 if (lowbyte >= LOWBYTE_BEGIN_2) {
76 --order;
77 }
78
79 return order;
80 }
81 }
+0
-195
src/org/mozilla/universalchardet/prober/sequence/BulgarianModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 public class BulgarianModel extends SequenceModel
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final float TYPICAL_POSITIVE_RATIO = 0.969392f;
45
46
47 ////////////////////////////////////////////////////////////////
48 // methods
49 ////////////////////////////////////////////////////////////////
50 public BulgarianModel(short[] charToOrderMap, String name)
51 {
52 super(
53 charToOrderMap,
54 bulgarianLangModel,
55 TYPICAL_POSITIVE_RATIO,
56 false,
57 name);
58 }
59
60
61 ////////////////////////////////////////////////////////////////
62 // constants continued
63 ////////////////////////////////////////////////////////////////
64 private static final byte[] bulgarianLangModel = new byte[] {
65 0,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,3,3,3,3,3,3,3,2,3,3,3,3,3,
66 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,0,3,3,3,2,2,3,2,2,1,2,2,
67 3,1,3,3,2,3,3,3,3,3,3,3,3,3,3,3,3,0,3,3,3,3,3,3,3,3,3,3,0,3,0,1,
68 0,0,0,0,0,0,0,0,0,0,1,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
69 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,2,3,3,3,3,3,3,3,3,0,3,1,0,
70 0,1,0,0,0,0,0,0,0,0,1,1,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
71 3,2,2,2,3,3,3,3,3,3,3,3,3,3,3,3,3,1,3,2,3,3,3,3,3,3,3,3,0,3,0,0,
72 0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
73 3,2,3,3,2,3,3,3,3,3,3,3,3,3,3,3,3,1,3,2,3,3,3,3,3,3,3,3,0,3,0,0,
74 0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
75 3,3,3,3,3,3,3,3,3,3,3,2,3,2,2,1,3,3,3,3,2,2,2,1,1,2,0,1,0,1,0,0,
76 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,1,
77 3,3,3,3,3,3,3,2,3,2,2,3,3,1,1,2,3,3,2,3,3,3,3,2,1,2,0,2,0,3,0,0,
78 0,0,0,0,0,0,0,1,0,0,2,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,1,
79 3,3,3,3,3,3,3,1,3,3,3,3,3,2,3,2,3,3,3,3,3,2,3,3,1,3,0,3,0,2,0,0,
80 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
81 3,3,3,3,3,3,3,3,1,3,3,2,3,3,3,1,3,3,2,3,2,2,2,0,0,2,0,2,0,2,0,0,
82 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,1,
83 3,3,3,3,3,3,3,3,3,0,3,3,3,2,2,3,3,3,1,2,2,3,2,1,1,2,0,2,0,0,0,0,
84 1,0,0,0,0,0,0,0,0,0,2,0,0,1,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
85 3,3,3,3,3,3,3,2,3,3,1,2,3,2,2,2,3,3,3,3,3,2,2,3,1,2,0,2,1,2,0,0,
86 0,0,0,0,0,0,0,0,0,0,3,0,0,1,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,1,
87 3,3,3,3,3,1,3,3,3,3,3,2,3,3,3,2,3,3,2,3,2,2,2,3,1,2,0,1,0,1,0,0,
88 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
89 3,3,3,3,3,3,3,3,3,3,3,1,1,1,2,2,1,3,1,3,2,2,3,0,0,1,0,1,0,1,0,0,
90 0,0,0,1,0,0,0,0,1,0,2,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
91 3,3,3,3,3,2,2,3,2,2,3,1,2,1,1,1,2,3,1,3,1,2,2,0,1,1,1,1,0,1,0,0,
92 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
93 3,3,3,3,3,1,3,2,2,3,3,1,2,3,1,1,3,3,3,3,1,2,2,1,1,1,0,2,0,2,0,1,
94 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
95 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,1,2,2,3,3,3,2,2,1,1,2,0,2,0,1,0,0,
96 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
97 3,0,1,2,1,3,3,2,3,3,3,3,3,2,3,2,1,0,3,1,2,1,2,1,2,3,2,1,0,1,0,0,
98 0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
99 1,1,1,2,3,3,3,3,3,3,3,3,3,3,3,3,0,0,3,1,3,3,2,3,3,2,2,2,0,1,0,0,
100 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
101 2,3,3,3,3,0,3,3,3,3,3,2,1,1,2,1,3,3,0,3,1,1,1,1,3,2,0,1,0,0,0,0,
102 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
103 3,3,2,2,2,3,3,3,3,3,3,3,3,3,3,3,1,1,3,1,3,3,2,3,2,2,2,3,0,2,0,0,
104 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
105 3,3,3,3,3,2,3,3,2,2,3,2,1,1,1,1,1,3,1,3,1,1,0,0,0,1,0,0,0,1,0,0,
106 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,
107 3,3,3,3,3,2,3,2,0,3,2,0,3,0,2,0,0,2,1,3,1,0,0,1,0,0,0,1,0,0,0,0,
108 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
109 3,3,3,3,2,1,1,1,1,2,1,1,2,1,1,1,2,2,1,2,1,1,1,0,1,1,0,1,0,1,0,0,
110 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
111 3,3,3,3,2,1,3,1,1,2,1,3,2,1,1,0,1,2,3,2,1,1,1,0,0,0,0,0,0,0,0,0,
112 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
113 2,3,3,3,3,2,2,1,0,1,0,0,1,0,0,0,2,1,0,3,0,0,1,0,0,0,0,0,0,0,0,0,
114 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
115 3,3,3,2,3,2,3,3,1,3,2,1,1,1,2,1,1,2,1,3,0,1,0,0,0,1,0,0,0,0,0,0,
116 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
117 3,1,1,2,2,3,3,2,3,2,2,2,3,1,2,2,1,1,2,1,1,2,2,0,1,1,0,1,0,2,0,0,
118 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
119 3,3,3,3,2,1,3,1,0,2,2,1,3,2,1,0,0,2,0,2,0,1,0,0,0,0,0,0,0,1,0,0,
120 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,
121 3,3,3,3,3,3,1,2,0,2,3,1,2,3,2,0,1,3,1,2,1,1,1,0,0,1,0,0,2,2,2,3,
122 2,2,2,2,1,2,1,1,2,2,1,1,2,0,1,1,1,0,0,1,1,0,0,1,1,0,0,0,1,1,0,1,
123 3,3,3,3,3,2,1,2,2,1,2,0,2,0,1,0,1,2,1,2,1,1,0,0,0,1,0,1,0,0,0,0,
124 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,1,
125 3,3,2,3,3,1,1,3,1,0,3,2,1,0,0,0,1,2,0,2,0,1,0,0,0,1,0,1,2,1,2,2,
126 1,1,1,1,1,1,1,2,2,2,1,1,1,1,1,1,1,0,1,2,1,1,1,0,0,0,0,0,1,1,0,0,
127 3,1,0,1,0,2,3,2,2,2,3,2,2,2,2,2,1,0,2,1,2,1,1,1,0,1,2,1,2,2,2,1,
128 1,1,2,2,2,2,1,2,1,1,0,1,2,1,2,2,2,1,1,1,0,1,1,1,1,2,0,1,0,0,0,0,
129 2,3,2,3,3,0,0,2,1,0,2,1,0,0,0,0,2,3,0,2,0,0,0,0,0,1,0,0,2,0,1,2,
130 2,1,2,1,2,2,1,1,1,2,1,1,1,0,1,2,2,1,1,1,1,1,0,1,1,1,0,0,1,2,0,0,
131 3,3,2,2,3,0,2,3,1,1,2,0,0,0,1,0,0,2,0,2,0,0,0,1,0,1,0,1,2,0,2,2,
132 1,1,1,1,2,1,0,1,2,2,2,1,1,1,1,1,1,1,0,1,1,1,0,0,0,0,0,0,1,1,0,0,
133 2,3,2,3,3,0,0,3,0,1,1,0,1,0,0,0,2,2,1,2,0,0,0,0,0,0,0,0,2,0,1,2,
134 2,2,1,1,1,1,1,2,2,2,1,0,2,0,1,0,1,0,0,1,0,1,0,0,1,0,0,0,0,1,0,0,
135 3,3,3,3,2,2,2,2,2,0,2,1,1,1,1,2,1,2,1,1,0,2,0,1,0,1,0,0,2,0,1,2,
136 1,1,1,1,1,1,1,2,2,1,1,0,2,0,1,0,2,0,0,1,1,1,0,0,2,0,0,0,1,1,0,0,
137 2,3,3,3,3,1,0,0,0,0,0,0,0,0,0,0,2,0,0,1,1,0,0,0,0,0,0,1,2,0,1,2,
138 2,2,2,1,1,2,1,1,2,2,2,1,2,0,1,1,1,1,1,1,0,1,1,1,1,0,0,1,1,1,0,0,
139 2,3,3,3,3,0,2,2,0,2,1,0,0,0,1,1,1,2,0,2,0,0,0,3,0,0,0,0,2,0,2,2,
140 1,1,1,2,1,2,1,1,2,2,2,1,2,0,1,1,1,0,1,1,1,1,0,2,1,0,0,0,1,1,0,0,
141 2,3,3,3,3,0,2,1,0,0,2,0,0,0,0,0,1,2,0,2,0,0,0,0,0,0,0,0,2,0,1,2,
142 1,1,1,2,1,1,1,1,2,2,2,0,1,0,1,1,1,0,0,1,1,1,0,0,1,0,0,0,0,1,0,0,
143 3,3,2,2,3,0,1,0,1,0,0,0,0,0,0,0,1,1,0,3,0,0,0,0,0,0,0,0,1,0,2,2,
144 1,1,1,1,1,2,1,1,2,2,1,2,2,1,0,1,1,1,1,1,0,1,0,0,1,0,0,0,1,1,0,0,
145 3,1,0,1,0,2,2,2,2,3,2,1,1,1,2,3,0,0,1,0,2,1,1,0,1,1,1,1,2,1,1,1,
146 1,2,2,1,2,1,2,2,1,1,0,1,2,1,2,2,1,1,1,0,0,1,1,1,2,1,0,1,0,0,0,0,
147 2,1,0,1,0,3,1,2,2,2,2,1,2,2,1,1,1,0,2,1,2,2,1,1,2,1,1,0,2,1,1,1,
148 1,2,2,2,2,2,2,2,1,2,0,1,1,0,2,1,1,1,1,1,0,0,1,1,1,1,0,1,0,0,0,0,
149 2,1,1,1,1,2,2,2,2,1,2,2,2,1,2,2,1,1,2,1,2,3,2,2,1,1,1,1,0,1,0,0,
150 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
151 2,2,2,3,2,0,1,2,0,1,2,1,1,0,1,0,1,2,1,2,0,0,0,1,1,0,0,0,1,0,0,2,
152 1,1,0,0,1,1,0,1,1,1,1,0,2,0,1,1,1,0,0,1,1,0,0,0,0,1,0,0,0,1,0,0,
153 2,0,0,0,0,1,2,2,2,2,2,2,2,1,2,1,1,1,1,1,1,1,0,1,1,1,1,1,2,1,1,1,
154 1,2,2,2,2,1,1,2,1,2,1,1,1,0,2,1,2,1,1,1,0,2,1,1,1,1,0,1,0,0,0,0,
155 3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,
156 1,1,0,1,0,1,1,1,1,1,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
157 2,2,2,3,2,0,0,0,0,1,0,0,0,0,0,0,1,1,0,2,0,0,0,0,0,0,0,0,1,0,1,2,
158 1,1,1,1,1,1,0,0,2,2,2,2,2,0,1,1,0,1,1,1,1,1,0,0,1,0,0,0,1,1,0,1,
159 2,3,1,2,1,0,1,1,0,2,2,2,0,0,1,0,0,1,1,1,1,0,0,0,0,0,0,0,1,0,1,2,
160 1,1,1,1,2,1,1,1,1,1,1,1,1,0,1,1,0,1,0,1,0,1,0,0,1,0,0,0,0,1,0,0,
161 2,2,2,2,2,0,0,2,0,0,2,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,2,0,2,2,
162 1,1,1,1,1,0,0,1,2,1,1,0,1,0,1,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,
163 1,2,2,2,2,0,0,2,0,1,1,0,0,0,1,0,0,2,0,2,0,0,0,0,0,0,0,0,0,0,1,1,
164 0,0,0,1,1,1,1,1,1,1,1,1,1,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,
165 1,2,2,3,2,0,0,1,0,0,1,0,0,0,0,0,0,1,0,2,0,0,0,1,0,0,0,0,0,0,0,2,
166 1,1,0,0,1,0,0,0,1,1,0,0,1,0,1,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,
167 2,1,2,2,2,1,2,1,2,2,1,1,2,1,1,1,0,1,1,1,1,2,0,1,0,1,1,1,1,0,1,1,
168 1,1,2,1,1,1,1,1,1,0,0,1,2,1,1,1,1,1,1,0,0,1,1,1,0,0,0,0,0,0,0,0,
169 1,0,0,1,3,1,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,
170 0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
171 2,2,2,2,1,0,0,1,0,2,0,0,0,0,0,1,1,1,0,1,0,0,0,0,0,0,0,0,2,0,0,1,
172 0,2,0,1,0,0,1,1,2,0,1,0,1,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,
173 1,2,2,2,2,0,1,1,0,2,1,0,1,1,1,0,0,1,0,2,0,1,0,0,0,0,0,0,0,0,0,1,
174 0,1,0,0,1,0,0,0,1,1,0,0,1,0,0,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,
175 2,2,2,2,2,0,0,1,0,0,0,1,0,1,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,1,
176 0,1,0,1,1,1,0,0,1,1,1,0,1,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,
177 2,0,1,0,0,1,2,1,1,1,1,1,1,2,2,1,0,0,1,0,1,0,0,0,0,1,1,1,1,0,0,0,
178 1,1,2,1,1,1,1,0,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
179 2,2,1,2,1,0,0,1,0,0,0,0,0,0,0,0,1,1,0,1,0,0,0,0,0,0,0,0,0,0,0,1,
180 0,0,0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
181 3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
182 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
183 1,0,0,1,2,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,
184 0,1,1,0,1,1,1,0,0,1,0,0,1,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,
185 1,0,1,0,0,1,1,1,1,1,1,1,1,1,1,1,0,0,1,0,2,0,0,2,0,1,0,0,1,0,0,1,
186 1,1,0,0,1,1,0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,
187 0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,
188 1,1,1,1,1,1,1,2,0,0,0,0,0,0,2,1,0,1,1,0,0,1,1,1,0,1,0,0,0,0,0,0,
189 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
190 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
191 1,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,0,1,1,0,1,1,1,1,1,0,1,0,0,
192 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
193 };
194 }
+0
-196
src/org/mozilla/universalchardet/prober/sequence/CyrillicModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 public class CyrillicModel extends SequenceModel
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final float TYPICAL_POSITIVE_RATIO = 0.976601f;
45
46
47 ////////////////////////////////////////////////////////////////
48 // methods
49 ////////////////////////////////////////////////////////////////
50 public CyrillicModel(short[] charToOrderMap, String name)
51 {
52 super(
53 charToOrderMap,
54 russianLangModel,
55 TYPICAL_POSITIVE_RATIO,
56 false,
57 name
58 );
59 }
60
61
62 ////////////////////////////////////////////////////////////////
63 // constants continued
64 ////////////////////////////////////////////////////////////////
65 private static final byte[] russianLangModel = new byte[] {
66 0,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,1,1,3,3,3,3,1,3,3,3,2,3,2,3,3,
67 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,0,3,2,2,2,2,2,0,0,2,
68 3,3,3,2,3,3,3,3,3,3,3,3,3,3,2,3,3,0,0,3,3,3,3,3,3,3,3,3,2,3,2,0,
69 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
70 3,3,3,2,2,3,3,3,3,3,3,3,3,3,2,3,3,0,0,3,3,3,3,3,3,3,3,2,3,3,1,0,
71 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
72 3,2,3,2,3,3,3,3,3,3,3,3,3,3,3,3,3,0,0,3,3,3,3,3,3,3,3,3,3,3,2,1,
73 0,0,0,0,0,0,0,2,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
74 3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,3,0,0,3,3,3,3,3,3,3,3,3,3,3,2,1,
75 0,0,0,0,0,1,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
76 3,3,3,3,3,3,3,3,2,2,2,3,1,3,3,1,3,3,3,3,2,2,3,0,2,2,2,3,3,2,1,0,
77 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,
78 3,3,3,3,3,3,2,3,3,3,3,3,2,2,3,2,3,3,3,2,1,2,2,0,1,2,2,2,2,2,2,0,
79 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,
80 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,2,3,0,2,2,3,3,2,1,2,0,
81 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,1,0,0,2,0,0,0,0,0,0,0,0,0,
82 3,3,3,3,3,3,2,3,3,1,2,3,2,2,3,2,3,3,3,3,2,2,3,0,3,2,2,3,1,1,1,0,
83 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
84 3,3,3,3,3,3,3,3,2,2,3,3,3,3,3,2,3,3,3,3,2,2,2,0,3,3,3,2,2,2,2,0,
85 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
86 3,3,3,3,3,3,3,3,3,3,2,3,2,3,3,3,3,3,3,2,3,2,2,0,1,3,2,1,2,2,1,0,
87 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,
88 3,3,3,3,3,3,3,3,3,3,3,2,1,1,3,0,1,1,1,1,2,1,1,0,2,2,2,1,2,0,1,0,
89 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
90 3,3,3,3,3,3,2,3,3,2,2,2,2,1,3,2,3,2,3,2,1,2,2,0,1,1,2,1,2,1,2,0,
91 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
92 3,3,3,3,3,3,3,3,3,3,3,3,2,2,3,2,3,3,3,2,2,2,2,0,2,2,2,2,3,1,1,0,
93 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,
94 3,2,3,2,2,3,3,3,3,3,3,3,3,3,1,3,2,0,0,3,3,3,3,2,3,3,3,3,2,3,2,0,
95 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
96 2,3,3,3,3,3,2,2,3,3,0,2,1,0,3,2,3,2,3,0,0,1,2,0,0,1,0,1,2,1,1,0,
97 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
98 3,0,3,0,2,3,3,3,3,2,3,3,3,3,1,2,2,0,0,2,3,2,2,2,3,2,3,2,2,3,0,0,
99 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
100 3,2,3,0,2,3,2,3,0,1,2,3,3,2,0,2,3,0,0,2,3,2,2,0,1,3,1,3,2,2,1,0,
101 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
102 3,1,3,0,2,3,3,3,3,3,3,3,3,2,1,3,2,0,0,2,2,3,3,3,2,3,3,0,2,2,0,0,
103 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
104 3,3,3,3,3,3,2,2,3,3,2,2,2,3,3,0,0,1,1,1,1,1,2,0,0,1,1,1,1,0,1,0,
105 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
106 3,3,3,3,3,3,2,2,3,3,3,3,3,3,3,0,3,2,3,3,2,3,2,0,2,1,0,1,1,0,1,0,
107 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,
108 3,3,3,3,3,3,2,3,3,3,2,2,2,2,3,1,3,2,3,1,1,2,1,0,2,2,2,2,1,3,1,0,
109 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,
110 2,2,3,3,3,3,3,1,2,2,1,3,1,0,3,0,0,3,0,0,0,1,1,0,1,2,1,0,0,0,0,0,
111 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
112 3,2,2,1,1,3,3,3,2,2,1,2,2,3,1,1,2,0,0,2,2,1,3,0,0,2,1,1,2,1,1,0,
113 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
114 3,2,3,3,3,3,1,2,2,2,1,2,1,3,3,1,1,2,1,2,1,2,2,0,2,0,0,1,1,0,1,0,
115 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
116 2,3,3,3,3,3,2,1,3,2,2,3,2,0,3,2,0,3,0,1,0,1,1,0,0,1,1,1,1,0,1,0,
117 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
118 3,3,2,3,3,3,2,2,2,3,3,1,2,1,2,1,0,1,0,1,1,0,1,0,0,2,1,1,1,0,1,0,
119 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,
120 3,1,1,2,1,2,3,3,2,2,1,2,2,3,0,2,1,0,0,2,2,3,2,1,2,2,2,2,2,3,1,0,
121 0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
122 3,3,3,3,3,1,1,0,1,1,2,2,1,1,3,0,0,1,3,1,1,1,0,0,0,1,0,1,1,0,0,0,
123 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
124 2,1,3,3,3,2,0,0,0,2,1,0,1,0,2,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
125 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
126 2,0,1,0,0,2,3,2,2,2,1,2,2,2,1,2,1,0,0,1,1,1,0,2,0,1,1,1,0,0,1,1,
127 1,0,0,0,0,0,1,2,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,
128 2,3,3,3,3,0,0,0,0,1,0,0,0,0,3,0,1,2,1,0,0,0,0,0,0,0,1,1,0,0,1,1,
129 1,0,1,0,1,2,0,0,1,1,2,1,0,1,1,1,1,0,1,1,1,1,0,1,0,0,1,0,0,1,1,0,
130 2,2,3,2,2,2,3,1,2,2,2,2,2,2,2,2,1,1,1,1,1,1,1,0,1,0,1,1,1,0,2,1,
131 1,1,1,1,1,1,1,1,2,1,1,1,1,1,1,1,1,1,1,0,1,0,1,1,0,1,1,1,0,1,1,0,
132 3,3,3,2,2,2,2,3,2,2,1,1,2,2,2,2,1,1,3,1,2,1,2,0,0,1,1,0,1,0,2,1,
133 1,1,1,1,1,2,1,0,1,1,1,1,0,1,0,0,1,1,0,0,1,0,1,0,0,1,0,0,0,1,1,0,
134 2,0,0,1,0,3,2,2,2,2,1,2,1,2,1,2,0,0,0,2,1,2,2,1,1,2,2,0,1,1,0,2,
135 1,1,1,1,1,0,1,1,1,2,1,1,1,2,1,0,1,2,1,1,1,1,0,1,1,1,0,0,1,0,0,1,
136 1,3,2,2,2,1,1,1,2,3,0,0,0,0,2,0,2,2,1,0,0,0,0,0,0,1,0,0,0,0,1,1,
137 1,0,1,1,0,1,0,1,1,0,1,1,0,2,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,1,1,0,
138 2,3,2,3,2,1,2,2,2,2,1,0,0,0,2,0,0,1,1,0,0,0,0,0,0,0,1,1,0,0,2,1,
139 1,1,2,1,0,2,0,0,1,0,1,0,0,1,0,0,1,1,0,1,1,0,0,0,0,0,1,0,0,0,0,0,
140 3,0,0,1,0,2,2,2,3,2,2,2,2,2,2,2,0,0,0,2,1,2,1,1,1,2,2,0,0,0,1,2,
141 1,1,1,1,1,0,1,2,1,1,1,1,1,1,1,0,1,1,1,1,1,1,0,1,1,1,1,1,1,0,0,1,
142 2,3,2,3,3,2,0,1,1,1,0,0,1,0,2,0,1,1,3,1,0,0,0,0,0,0,0,1,0,0,2,1,
143 1,1,1,1,1,1,1,0,1,0,1,1,1,1,0,1,1,1,0,0,1,1,0,1,0,0,0,0,0,0,1,0,
144 2,3,3,3,3,1,2,2,2,2,0,1,1,0,2,1,1,1,2,1,0,1,1,0,0,1,0,1,0,0,2,0,
145 0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
146 2,3,3,3,2,0,0,1,1,2,2,1,0,0,2,0,1,1,3,0,0,1,0,0,0,0,0,1,0,1,2,1,
147 1,1,2,0,1,1,1,0,1,0,1,1,0,1,0,1,1,1,1,0,1,0,0,0,0,0,0,1,0,1,1,0,
148 1,3,2,3,2,1,0,0,2,2,2,0,1,0,2,0,1,1,1,0,1,0,0,0,3,0,1,1,0,0,2,1,
149 1,1,1,0,1,1,0,0,0,0,1,1,0,1,0,0,2,1,1,0,1,0,0,0,1,0,1,0,0,1,1,0,
150 3,1,2,1,1,2,2,2,2,2,2,1,2,2,1,1,0,0,0,2,2,2,0,0,0,1,2,1,0,1,0,1,
151 2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,2,1,1,1,0,1,0,1,1,0,1,1,1,0,0,1,
152 3,0,0,0,0,2,0,1,1,1,1,1,1,1,0,1,0,0,0,1,1,1,0,1,0,1,1,0,0,1,0,1,
153 1,1,0,0,1,0,0,0,1,0,1,1,0,0,1,0,1,0,1,0,0,0,0,1,0,0,0,1,0,0,0,1,
154 1,3,3,2,2,0,0,0,2,2,0,0,0,1,2,0,1,1,2,0,0,0,0,0,0,0,0,1,0,0,2,1,
155 0,1,1,0,0,1,1,0,0,0,1,1,0,1,1,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,1,0,
156 2,3,2,3,2,0,0,0,0,1,1,0,0,0,2,0,2,0,2,0,0,0,0,0,1,0,0,1,0,0,1,1,
157 1,1,2,0,1,2,1,0,1,1,2,1,1,1,1,1,2,1,1,0,1,0,0,1,1,1,1,1,0,1,1,0,
158 1,3,2,2,2,1,0,0,2,2,1,0,1,2,2,0,0,1,0,0,0,0,0,0,0,0,0,1,0,0,1,1,
159 0,0,1,1,0,1,1,0,0,1,1,0,1,1,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,
160 1,0,0,1,0,2,3,1,2,2,2,2,2,2,1,1,0,0,0,1,0,1,0,2,1,1,1,0,0,0,0,1,
161 1,1,0,1,1,0,1,1,1,1,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,
162 2,0,2,0,0,1,0,3,2,1,2,1,2,2,0,1,0,0,0,2,1,0,0,2,1,1,1,1,0,2,0,2,
163 2,1,1,1,1,1,1,1,1,1,1,1,1,2,1,0,1,1,1,1,0,0,0,1,1,1,1,0,1,0,0,1,
164 1,2,2,2,2,1,0,0,1,0,0,0,0,0,2,0,1,1,1,1,0,0,0,0,1,0,1,2,0,0,2,0,
165 1,0,1,1,1,2,1,0,1,0,1,1,0,0,1,0,1,1,1,0,1,0,0,0,1,0,0,1,0,1,1,0,
166 2,1,2,2,2,0,3,0,1,1,0,0,0,0,2,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
167 0,0,0,1,1,1,0,0,1,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,
168 1,2,2,3,2,2,0,0,1,1,2,0,1,2,1,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,0,1,
169 0,1,1,0,0,1,1,0,0,1,1,0,0,1,1,0,1,1,0,0,1,0,0,0,0,0,0,0,0,1,1,0,
170 2,2,1,1,2,1,2,2,2,2,2,1,2,2,0,1,0,0,0,1,2,2,2,1,2,1,1,1,1,1,2,1,
171 1,1,1,1,1,1,1,1,1,1,0,0,1,1,1,0,1,1,1,0,0,0,0,1,1,1,0,1,1,0,0,1,
172 1,2,2,2,2,0,1,0,2,2,0,0,0,0,2,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,2,0,
173 0,0,1,0,0,1,0,0,0,0,1,0,1,1,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,
174 0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,
175 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
176 1,2,2,2,2,0,0,0,2,2,2,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,1,
177 0,1,1,0,0,1,1,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
178 1,2,2,2,2,0,0,0,0,1,0,0,1,1,2,0,0,0,0,1,0,1,0,0,1,0,0,2,0,0,0,1,
179 0,0,1,0,0,1,0,0,0,1,1,0,0,0,0,0,1,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,
180 1,2,2,2,1,1,2,0,2,1,1,1,1,0,2,2,0,0,0,0,0,0,0,0,0,1,1,0,0,0,1,1,
181 0,0,1,0,1,1,0,0,0,0,1,0,0,0,0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,
182 1,0,2,1,2,0,0,0,0,0,1,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,
183 0,0,1,0,1,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,
184 1,0,0,0,0,2,0,1,2,1,0,1,1,1,0,1,0,0,0,1,0,1,0,0,1,0,1,0,0,0,0,1,
185 0,0,0,0,0,1,0,0,1,1,0,0,1,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,
186 2,2,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
187 1,0,0,0,1,0,0,0,1,1,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,
188 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
189 1,1,1,0,1,0,1,0,0,1,1,1,1,0,0,0,1,0,0,0,0,1,0,0,0,1,0,1,0,0,0,0,
190 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
191 1,1,0,1,1,0,1,0,1,0,0,0,0,1,1,0,1,1,0,0,0,0,0,1,0,1,1,0,1,0,0,0,
192 0,1,1,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
193 0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,
194 };
195 }
+0
-196
src/org/mozilla/universalchardet/prober/sequence/GreekModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 public class GreekModel extends SequenceModel
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final float TYPICAL_POSITIVE_RATIO = 0.982851f;
45
46
47 ////////////////////////////////////////////////////////////////
48 // methods
49 ////////////////////////////////////////////////////////////////
50 public GreekModel(short[] charToOrderMap, String name)
51 {
52 super(
53 charToOrderMap,
54 greekLangModel,
55 TYPICAL_POSITIVE_RATIO,
56 false,
57 name
58 );
59 }
60
61
62 ////////////////////////////////////////////////////////////////
63 // constants continued
64 ////////////////////////////////////////////////////////////////
65 private static final byte[] greekLangModel = new byte[] {
66 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
67 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
68 0,0,3,2,2,3,3,3,3,3,3,3,3,1,3,3,3,0,2,2,3,3,0,3,0,3,2,0,3,3,3,0,
69 3,0,0,0,2,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
70 0,3,3,3,3,3,0,3,3,0,3,2,3,3,0,3,2,3,3,3,0,0,3,0,3,0,3,3,2,0,0,0,
71 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,
72 0,2,3,2,2,3,3,3,3,3,3,3,3,0,3,3,3,3,0,2,3,3,0,3,3,3,3,2,3,3,3,0,
73 2,0,0,0,2,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
74 0,2,3,3,2,3,3,3,3,3,3,3,3,3,3,3,3,0,2,1,3,3,3,3,2,3,3,2,3,3,2,0,
75 0,0,0,0,2,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
76 0,3,3,3,3,0,3,3,3,3,3,3,0,3,3,0,3,3,3,3,3,3,3,3,3,3,0,3,2,3,3,0,
77 2,0,1,0,2,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,
78 0,3,3,3,3,3,2,3,0,0,0,0,3,3,0,3,1,3,3,3,0,3,3,0,3,3,3,3,0,0,0,0,
79 2,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
80 0,3,3,3,3,3,0,3,0,3,3,3,3,3,0,3,2,2,2,3,0,2,3,3,3,3,3,2,3,3,0,0,
81 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
82 0,3,3,3,3,3,3,2,2,2,3,3,3,3,0,3,1,3,3,3,3,2,3,3,3,3,3,3,3,2,2,0,
83 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
84 0,3,3,3,3,3,2,0,3,0,0,0,3,3,2,3,3,3,3,3,0,0,3,2,3,0,2,3,0,0,0,0,
85 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
86 0,3,0,3,3,3,3,0,0,3,3,0,2,3,0,3,0,3,3,3,0,0,3,0,3,0,2,2,3,3,0,0,
87 0,0,1,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
88 0,3,3,3,3,3,2,0,3,2,3,3,3,3,0,3,3,3,3,3,0,3,3,2,3,2,3,3,2,0,0,0,
89 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
90 0,3,3,2,3,2,3,3,3,3,3,3,0,2,3,2,3,2,2,2,3,2,3,3,2,3,0,2,2,2,3,0,
91 2,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
92 0,0,3,0,0,0,3,3,3,2,3,3,0,0,3,0,3,0,0,0,3,2,0,3,0,3,0,0,2,0,2,0,
93 0,0,0,0,2,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
94 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
95 0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
96 0,3,3,3,3,0,3,3,3,3,3,3,0,3,3,0,3,0,0,0,3,3,0,3,3,3,0,0,1,2,3,0,
97 3,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
98 0,3,3,3,3,3,2,0,0,3,2,2,3,3,0,3,3,3,3,3,2,1,3,0,3,2,3,3,2,1,0,0,
99 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
100 0,0,3,3,0,2,3,3,3,3,3,3,0,0,3,0,3,0,0,0,3,3,0,3,2,3,0,0,3,3,3,0,
101 3,0,0,0,2,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
102 0,3,3,3,3,0,3,3,3,3,3,3,0,0,3,0,3,0,0,0,3,2,0,3,2,3,0,0,3,2,3,0,
103 2,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
104 0,0,3,1,2,2,3,3,3,3,3,3,0,2,3,0,3,0,0,0,3,3,0,3,0,2,0,0,2,3,1,0,
105 2,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
106 0,3,0,3,3,3,3,0,3,0,3,3,2,3,0,3,3,3,3,3,3,0,3,3,3,0,2,3,0,0,3,0,
107 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
108 0,3,0,3,3,3,0,0,3,0,0,0,3,3,0,3,0,2,3,3,0,0,3,0,3,0,3,3,0,0,0,0,
109 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
110 0,0,3,0,0,0,3,3,3,3,3,3,0,0,3,0,2,0,0,0,3,3,0,3,0,3,0,0,2,0,2,0,
111 0,0,0,0,1,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
112 0,3,3,3,3,3,3,0,3,0,2,0,3,2,0,3,2,3,2,3,0,0,3,2,3,2,3,3,0,0,0,0,
113 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
114 0,0,3,0,0,2,3,3,3,3,3,0,0,0,3,0,2,1,0,0,3,2,2,2,0,3,0,0,2,2,0,0,
115 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
116 0,3,0,3,3,3,2,0,3,0,3,0,3,3,0,2,1,2,3,3,0,0,3,0,3,0,3,3,0,0,0,0,
117 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
118 0,2,3,3,3,0,3,3,3,3,3,3,0,2,3,0,3,0,0,0,2,1,0,2,2,3,0,0,2,2,2,0,
119 0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
120 0,0,3,0,0,2,3,3,3,2,3,0,0,1,3,0,2,0,0,0,0,3,0,1,0,2,0,0,1,1,1,0,
121 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
122 0,3,3,3,3,3,1,0,3,0,0,0,3,2,0,3,2,3,3,3,0,0,3,0,3,2,2,2,1,0,0,0,
123 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
124 0,3,0,3,3,3,0,0,3,0,0,0,0,2,0,2,3,3,2,2,2,2,3,0,2,0,2,2,0,0,0,0,
125 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
126 0,3,3,3,3,2,0,0,0,0,0,0,2,3,0,2,0,2,3,2,0,0,3,0,3,0,3,1,0,0,0,0,
127 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
128 0,0,0,0,0,0,3,2,3,3,2,2,3,0,2,0,3,0,0,0,2,0,0,0,0,1,2,0,2,0,2,0,
129 0,2,0,2,0,2,2,0,0,1,0,2,2,2,0,2,2,2,0,2,2,2,0,0,2,0,0,1,0,0,0,0,
130 0,2,0,3,3,2,0,0,0,0,0,0,1,3,0,2,0,2,2,2,0,0,2,0,3,0,0,2,0,0,0,0,
131 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
132 0,3,0,2,3,2,0,2,2,0,2,0,2,2,0,2,0,2,2,2,0,0,0,0,0,0,2,3,0,0,0,2,
133 0,1,2,0,0,0,0,2,2,0,0,0,2,1,0,2,2,0,0,0,0,0,0,1,0,2,0,0,0,0,0,0,
134 0,0,2,1,0,2,3,2,2,3,2,3,2,0,0,3,3,3,0,0,3,2,0,0,0,1,1,0,2,0,2,2,
135 0,2,0,2,0,2,2,0,0,2,0,2,2,2,0,2,2,2,2,0,0,2,0,0,0,2,0,1,0,0,0,0,
136 0,3,0,3,3,2,2,0,3,0,0,0,2,2,0,2,2,2,1,2,0,0,1,2,2,0,0,3,0,0,0,2,
137 0,1,2,0,0,0,1,2,0,0,0,0,0,0,0,2,2,0,1,0,0,2,0,0,0,2,0,0,0,0,0,0,
138 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
139 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
140 0,2,3,3,2,2,0,0,0,2,0,2,3,3,0,2,0,0,0,0,0,0,2,2,2,0,2,2,0,2,0,2,
141 0,2,2,0,0,2,2,2,2,1,0,0,2,2,0,2,0,0,2,0,0,0,0,0,0,2,0,0,0,0,0,0,
142 0,2,0,3,2,3,0,0,0,3,0,0,2,2,0,2,0,2,2,2,0,0,2,0,0,0,0,0,0,0,0,2,
143 0,0,2,2,0,0,2,2,2,0,0,0,0,0,0,2,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,
144 0,0,2,0,0,3,2,0,2,2,2,2,2,0,0,0,2,0,0,0,0,2,0,1,0,0,2,0,1,0,0,0,
145 0,2,2,2,0,2,2,0,1,2,0,2,2,2,0,2,2,2,2,1,2,2,0,0,2,0,0,0,0,0,0,0,
146 0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,
147 0,2,0,2,0,2,2,0,0,0,0,1,2,1,0,0,2,2,0,0,2,0,0,0,0,0,0,0,0,0,0,0,
148 0,0,0,3,2,3,0,0,2,0,0,0,2,2,0,2,0,0,0,1,0,0,2,0,2,0,2,2,0,0,0,0,
149 0,0,2,0,0,0,0,2,2,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,
150 0,2,2,3,2,2,0,0,0,0,0,0,1,3,0,2,0,2,2,0,0,0,1,0,2,0,0,0,0,0,0,0,
151 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
152 0,2,0,2,0,3,2,0,2,0,0,0,0,0,0,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,
153 0,0,2,0,0,0,0,1,1,0,0,2,1,2,0,2,2,0,1,0,0,1,0,0,0,2,0,0,0,0,0,0,
154 0,3,0,2,2,2,0,0,2,0,0,0,2,0,0,0,2,3,0,2,0,0,0,0,0,0,2,2,0,0,0,2,
155 0,1,2,0,0,0,1,2,2,1,0,0,0,2,0,0,2,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,
156 0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
157 0,2,1,2,0,2,2,0,2,0,0,2,0,0,0,0,1,2,1,0,2,1,0,0,0,0,0,0,0,0,0,0,
158 0,0,2,0,0,0,3,1,2,2,0,2,0,0,0,0,2,0,0,0,2,0,0,3,0,0,0,0,2,2,2,0,
159 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
160 0,2,1,0,2,0,1,2,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,1,0,0,0,0,0,0,2,
161 0,2,2,0,0,2,2,2,2,2,0,1,2,0,0,0,2,2,0,1,0,2,0,0,2,2,0,0,0,0,0,0,
162 0,0,0,0,1,0,0,0,0,0,0,0,3,0,0,2,0,0,0,0,0,0,0,0,2,0,2,0,0,0,0,2,
163 0,1,2,0,0,0,0,2,2,1,0,1,0,1,0,2,2,2,1,0,0,0,0,0,0,1,0,0,0,0,0,0,
164 0,2,0,1,2,0,0,0,0,0,0,0,0,0,0,2,0,0,2,2,0,0,0,0,1,0,0,0,0,0,0,2,
165 0,2,2,0,0,0,0,2,2,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,2,0,0,2,0,0,0,
166 0,2,2,2,2,0,0,0,3,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,2,0,0,0,0,0,0,1,
167 0,0,2,0,0,0,0,1,2,0,0,0,0,0,0,2,2,1,1,0,0,0,0,0,0,1,0,0,0,0,0,0,
168 0,2,0,2,2,2,0,0,2,0,0,0,0,0,0,0,2,2,2,0,0,0,2,0,0,0,0,0,0,0,0,2,
169 0,0,1,0,0,0,0,2,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,
170 0,3,0,2,0,0,0,0,0,0,0,0,2,0,0,0,0,0,2,0,0,0,0,0,0,0,2,0,0,0,0,2,
171 0,0,2,0,0,0,0,2,2,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
172 0,2,0,2,2,1,0,0,0,0,0,0,2,0,0,2,0,2,2,2,0,0,0,0,0,0,2,0,0,0,0,2,
173 0,0,2,0,0,2,0,2,2,0,0,0,0,2,0,2,0,0,0,0,0,2,0,0,0,2,0,0,0,0,0,0,
174 0,0,3,0,0,0,2,2,0,2,2,0,0,0,0,0,2,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,
175 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
176 0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
177 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,2,0,0,0,0,0,
178 0,2,2,2,2,2,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,1,
179 0,0,0,0,0,0,0,2,1,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
180 0,0,0,0,0,0,0,2,2,0,0,0,0,0,2,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,
181 0,2,0,0,0,2,0,0,0,0,0,1,0,0,0,0,2,2,0,0,0,1,0,0,0,0,0,0,0,0,0,0,
182 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
183 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,2,0,0,0,
184 0,2,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,
185 0,0,1,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
186 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
187 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,2,0,2,0,0,0,
188 0,0,0,0,0,0,0,0,2,1,0,0,0,0,0,0,2,0,0,0,1,2,0,0,0,0,0,0,0,0,0,0,
189 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
190 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
191 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
192 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
193 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
194 };
195 }
+0
-219
src/org/mozilla/universalchardet/prober/sequence/HebrewModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Simon Montagu <smontagu@smontagu.org>
17 * Portions created by the Initial Developer are Copyright (C) 2005
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shoshannah Forbes <xslf@xslf.com>
22 * Shy Shalom <shooshX@gmail.com>
23 * Kohei TAKETA <k-tak@void.in> (Java port)
24 *
25 * Alternatively, the contents of this file may be used under the terms of
26 * either the GNU General Public License Version 2 or later (the "GPL"), or
27 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
28 * in which case the provisions of the GPL or the LGPL are applicable instead
29 * of those above. If you wish to allow use of your version of this file only
30 * under the terms of either the GPL or the LGPL, and not to allow others to
31 * use your version of this file under the terms of the MPL, indicate your
32 * decision by deleting the provisions above and replace them with the notice
33 * and other provisions required by the GPL or the LGPL. If you do not delete
34 * the provisions above, a recipient may use your version of this file under
35 * the terms of any one of the MPL, the GPL or the LGPL.
36 *
37 * ***** END LICENSE BLOCK ***** */
38
39 package org.mozilla.universalchardet.prober.sequence;
40
41 import org.mozilla.universalchardet.Constants;
42
43
44 public class HebrewModel extends SequenceModel
45 {
46 ////////////////////////////////////////////////////////////////
47 // constants
48 ////////////////////////////////////////////////////////////////
49 public static final float TYPICAL_POSITIVE_RATIO = 0.984004f;
50
51
52 ////////////////////////////////////////////////////////////////
53 // methods
54 ////////////////////////////////////////////////////////////////
55 public HebrewModel()
56 {
57 super(
58 win1255CharToOrderMap,
59 hebrewLangModel,
60 TYPICAL_POSITIVE_RATIO,
61 false,
62 Constants.CHARSET_WINDOWS_1255);
63 }
64
65
66 ////////////////////////////////////////////////////////////////
67 // constants continued
68 ////////////////////////////////////////////////////////////////
69 private static final short[] win1255CharToOrderMap = new short[] {
70 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
71 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
72 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
73 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
74 253, 69, 91, 79, 80, 92, 89, 97, 90, 68,111,112, 82, 73, 95, 85, //40
75 78,121, 86, 71, 67,102,107, 84,114,103,115,253,253,253,253,253, //50
76 253, 50, 74, 60, 61, 42, 76, 70, 64, 53,105, 93, 56, 65, 54, 49, //60
77 66,110, 51, 43, 44, 63, 81, 77, 98, 75,108,253,253,253,253,253, //70
78 124,202,203,204,205, 40, 58,206,207,208,209,210,211,212,213,214,
79 215, 83, 52, 47, 46, 72, 32, 94,216,113,217,109,218,219,220,221,
80 34,116,222,118,100,223,224,117,119,104,125,225,226, 87, 99,227,
81 106,122,123,228, 55,229,230,101,231,232,120,233, 48, 39, 57,234,
82 30, 59, 41, 88, 33, 37, 36, 31, 29, 35,235, 62, 28,236,126,237,
83 238, 38, 45,239,240,241,242,243,127,244,245,246,247,248,249,250,
84 9, 8, 20, 16, 3, 2, 24, 14, 22, 1, 25, 15, 4, 11, 6, 23,
85 12, 19, 13, 26, 18, 27, 21, 17, 7, 10, 5,251,252,128, 96,253,
86 };
87
88 private static final byte[] hebrewLangModel = new byte[] {
89 0,3,3,3,3,3,3,3,3,3,3,2,3,3,3,3,3,3,3,3,3,3,3,2,3,2,1,2,0,1,0,0,
90 3,0,3,1,0,0,1,3,2,0,1,1,2,0,2,2,2,1,1,1,1,2,1,1,1,2,0,0,2,2,0,1,
91 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,2,2,
92 1,2,1,2,1,2,0,0,2,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,
93 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,2,
94 1,2,1,3,1,1,0,0,2,0,0,0,1,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,
95 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,1,0,1,2,2,1,3,
96 1,2,1,1,2,2,0,0,2,2,0,0,0,0,1,0,1,0,0,0,1,0,0,0,0,0,0,1,0,1,1,0,
97 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,3,2,2,2,2,3,2,
98 1,2,1,2,2,2,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,
99 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,3,2,3,2,2,3,2,2,2,1,2,2,2,2,
100 1,2,1,1,2,2,0,1,2,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,1,0,
101 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,0,2,2,2,2,2,
102 0,2,0,2,2,2,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,
103 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,0,2,2,2,
104 0,2,1,2,2,2,0,0,2,1,0,0,0,0,1,0,1,0,0,0,0,0,0,2,0,0,0,0,0,0,1,0,
105 3,3,3,3,3,3,3,3,3,3,3,2,3,3,3,3,3,3,3,3,3,3,3,3,3,2,1,2,3,2,2,2,
106 1,2,1,2,2,2,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,1,0,
107 3,3,3,3,3,3,3,3,3,2,3,3,3,2,3,3,3,3,3,3,3,3,3,3,3,3,3,1,0,2,0,2,
108 0,2,1,2,2,2,0,0,1,2,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,2,0,0,1,0,
109 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,2,3,2,2,3,2,1,2,1,1,1,
110 0,1,1,1,1,1,3,0,1,0,0,0,0,2,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
111 3,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1,0,1,1,0,0,1,0,0,1,0,0,0,0,
112 0,0,1,0,0,0,0,0,2,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
113 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,2,2,2,2,2,
114 0,2,0,1,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
115 3,3,3,3,3,3,3,3,3,2,3,3,3,2,1,2,3,3,2,3,3,3,3,2,3,2,1,2,0,2,1,2,
116 0,2,0,2,2,2,0,0,1,2,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,
117 3,3,3,3,3,3,3,3,3,2,3,3,3,1,2,2,3,3,2,3,2,3,2,2,3,1,2,2,0,2,2,2,
118 0,2,1,2,2,2,0,0,1,2,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,1,0,0,1,0,
119 3,3,3,3,3,3,3,3,3,3,3,3,3,2,3,3,3,2,3,3,2,2,2,3,3,3,3,1,3,2,2,2,
120 0,2,0,1,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
121 3,3,3,3,3,3,3,3,3,3,3,3,3,3,2,2,3,3,3,2,3,2,2,2,1,2,2,0,2,2,2,2,
122 0,2,0,2,2,2,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
123 3,3,3,3,3,3,3,3,3,3,3,2,3,3,3,1,3,2,3,3,2,3,3,2,2,1,2,2,2,2,2,2,
124 0,2,1,2,1,2,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,
125 3,3,3,3,3,3,2,3,2,3,3,2,3,3,3,3,2,3,2,3,3,3,3,3,2,2,2,2,2,2,2,1,
126 0,2,0,1,2,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,
127 3,3,3,3,3,3,3,3,3,2,1,2,3,3,3,3,3,3,3,2,3,2,3,2,1,2,3,0,2,1,2,2,
128 0,2,1,1,2,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,2,0,
129 3,3,3,3,3,3,3,3,3,2,3,3,3,3,2,1,3,1,2,2,2,1,2,3,3,1,2,1,2,2,2,2,
130 0,1,1,1,1,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,2,0,0,0,0,0,0,0,0,
131 3,3,3,3,3,3,3,3,3,3,0,2,3,3,3,1,3,3,3,1,2,2,2,2,1,1,2,2,2,2,2,2,
132 0,2,0,1,1,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,
133 3,3,3,3,3,3,2,3,3,3,2,2,3,3,3,2,1,2,3,2,3,2,2,2,2,1,2,1,1,1,2,2,
134 0,2,1,1,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
135 3,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,0,0,0,1,0,0,0,0,0,
136 1,0,1,0,0,0,0,0,2,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
137 3,3,3,3,3,2,3,3,2,3,1,2,2,2,2,3,2,3,1,1,2,2,1,2,2,1,1,0,2,2,2,2,
138 0,1,0,1,2,2,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,
139 3,0,0,1,1,0,1,0,0,1,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,2,2,0,
140 0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
141 3,0,1,0,1,0,1,1,0,1,1,0,0,0,1,1,0,1,1,1,0,0,0,0,0,0,1,0,0,0,0,0,
142 0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
143 3,0,0,0,1,1,0,1,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,
144 0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,
145 3,2,2,1,2,2,2,2,2,2,2,1,2,2,1,2,2,1,1,1,1,1,1,1,1,2,1,1,0,3,3,3,
146 0,3,0,2,2,2,2,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
147 2,2,2,3,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,1,2,2,1,2,2,2,1,1,1,2,0,1,
148 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
149 2,2,2,2,2,2,2,2,2,2,2,1,2,2,2,2,2,2,2,2,2,2,2,0,2,2,0,0,0,0,0,0,
150 0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
151 2,3,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,1,2,1,0,2,1,0,
152 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
153 3,1,1,1,1,1,1,1,1,1,1,0,0,1,1,1,1,0,1,1,1,1,0,0,0,0,0,0,0,0,0,0,
154 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,
155 0,3,1,1,2,2,2,2,2,1,2,2,2,1,1,2,2,2,2,2,2,2,1,2,2,1,0,1,1,1,1,0,
156 0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
157 3,2,1,1,1,1,2,1,1,2,1,0,1,1,1,1,1,1,1,1,1,1,1,0,1,0,0,0,0,0,0,0,
158 0,0,2,0,0,0,0,0,0,0,0,1,1,0,0,0,0,1,1,0,0,1,1,0,0,0,0,0,0,1,0,0,
159 2,1,1,2,2,2,2,2,2,2,2,2,2,2,1,2,2,2,2,2,1,2,1,2,1,1,1,1,0,0,0,0,
160 0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
161 1,2,1,2,2,2,2,2,2,2,2,2,2,1,2,1,2,1,1,2,1,1,1,2,1,2,1,2,0,1,0,1,
162 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
163 0,3,1,2,2,2,1,2,2,2,2,2,2,2,2,1,2,1,1,1,1,1,1,2,1,2,1,1,0,1,0,1,
164 0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
165 2,1,2,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,2,2,
166 0,2,0,1,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,
167 3,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,
168 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
169 2,1,1,1,1,1,1,1,0,1,1,0,1,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,
170 0,0,0,0,0,0,0,0,2,0,1,1,1,0,1,0,0,0,1,1,0,1,1,0,0,0,0,0,1,1,0,0,
171 0,1,1,1,2,1,2,2,2,0,2,0,2,0,1,1,2,1,1,1,1,2,1,0,1,1,0,0,0,0,0,0,
172 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
173 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,
174 1,0,1,0,0,0,0,0,1,0,1,2,2,0,1,0,0,1,1,2,2,1,2,0,2,0,0,0,1,2,0,1,
175 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
176 0,0,0,0,0,0,0,0,2,0,2,1,2,0,2,0,0,1,1,1,1,1,1,0,1,0,0,0,1,0,0,1,
177 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
178 0,0,1,0,0,0,0,0,1,0,2,1,1,0,1,0,0,1,1,1,2,2,0,0,1,0,0,0,1,0,0,1,
179 1,1,2,1,0,1,1,1,0,1,0,1,1,1,1,0,0,0,1,0,1,0,0,0,0,0,0,0,0,2,2,1,
180 0,2,0,1,2,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
181 2,1,0,0,1,0,1,1,1,1,0,0,0,0,0,1,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,
182 0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
183 1,1,1,1,1,1,1,1,1,2,1,0,1,1,1,1,1,1,1,1,1,1,1,0,1,0,0,0,0,0,0,0,
184 0,0,0,0,0,0,0,0,0,0,1,1,1,0,0,0,0,1,1,1,0,1,1,0,1,0,0,0,1,1,0,1,
185 2,0,1,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
186 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
187 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
188 0,0,0,0,0,0,0,0,1,0,1,1,1,0,1,0,0,1,1,2,1,1,2,0,1,0,0,0,1,1,0,1,
189 1,0,0,1,0,0,1,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
190 0,0,0,0,0,0,0,0,1,0,1,1,2,0,1,0,0,0,0,2,1,1,2,0,2,0,0,0,1,1,0,1,
191 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
192 0,0,0,0,0,0,0,0,1,0,2,1,1,0,1,0,0,2,2,1,2,1,1,0,1,0,0,0,1,1,0,1,
193 2,0,1,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
194 0,0,0,0,0,0,0,0,0,0,1,2,2,0,0,0,0,0,1,1,0,1,0,0,1,0,0,0,0,1,0,1,
195 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
196 0,0,0,0,0,0,0,0,0,0,1,2,2,0,0,0,0,2,1,1,1,0,2,1,1,0,0,0,2,1,0,1,
197 1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
198 0,0,0,0,0,0,0,0,1,0,1,1,2,0,1,0,0,1,1,0,2,1,1,0,1,0,0,0,1,1,0,1,
199 2,2,1,1,1,0,1,1,0,1,1,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,
200 0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
201 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
202 0,0,0,0,0,0,0,0,1,0,2,1,1,0,1,0,0,1,1,0,1,2,1,0,2,0,0,0,1,1,0,1,
203 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
204 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
205 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
206 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,
207 0,1,0,0,2,0,2,1,1,0,1,0,1,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,
208 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
209 1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
210 0,0,0,0,0,0,0,0,1,0,1,1,2,0,1,0,0,1,1,1,0,1,0,0,1,0,0,0,1,0,0,1,
211 1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
212 1,0,0,0,0,0,0,0,1,0,1,1,0,0,1,0,0,2,1,1,1,1,1,0,1,0,0,0,0,1,0,1,
213 0,1,1,1,2,1,1,1,1,0,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1,0,0,0,0,0,0,0,
214 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
215 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
216 0,0,0,0,0,0,0,0,0,0,1,2,1,0,0,0,0,0,1,1,1,1,1,0,1,0,0,0,1,1,0,0,
217 };
218 }
+0
-75
src/org/mozilla/universalchardet/prober/sequence/Ibm855Model.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Ibm855Model extends CyrillicModel
43 {
44 ////////////////////////////////////////////////////////////////
45 // methods
46 ////////////////////////////////////////////////////////////////
47 public Ibm855Model()
48 {
49 super(ibm855CharToOrderMap, Constants.CHARSET_IBM855);
50 }
51
52
53 ////////////////////////////////////////////////////////////////
54 // constants
55 ////////////////////////////////////////////////////////////////
56 private static final short[] ibm855CharToOrderMap = new short[] {
57 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
58 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
59 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
60 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
61 253,142,143,144,145,146,147,148,149,150,151,152, 74,153, 75,154, //40
62 155,156,157,158,159,160,161,162,163,164,165,253,253,253,253,253, //50
63 253, 71,172, 66,173, 65,174, 76,175, 64,176,177, 77, 72,178, 69, //60
64 67,179, 78, 73,180,181, 79,182,183,184,185,253,253,253,253,253, //70
65 191,192,193,194, 68,195,196,197,198,199,200,201,202,203,204,205,
66 206,207,208,209,210,211,212,213,214,215,216,217, 27, 59, 54, 70,
67 3, 37, 21, 44, 28, 58, 13, 41, 2, 48, 39, 53, 19, 46,218,219,
68 220,221,222,223,224, 26, 55, 4, 42,225,226,227,228, 23, 60,229,
69 230,231,232,233,234,235, 11, 36,236,237,238,239,240,241,242,243,
70 8, 49, 12, 38, 5, 31, 1, 34, 15,244,245,246,247, 35, 16,248,
71 43, 9, 45, 7, 32, 6, 40, 14, 52, 24, 56, 10, 33, 17, 61,249,
72 250, 18, 62, 20, 51, 25, 57, 30, 47, 29, 63, 22, 50,251,252,255,
73 };
74 }
+0
-75
src/org/mozilla/universalchardet/prober/sequence/Ibm866Model.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Ibm866Model extends CyrillicModel
43 {
44 ////////////////////////////////////////////////////////////////
45 // methods
46 ////////////////////////////////////////////////////////////////
47 public Ibm866Model()
48 {
49 super(ibm866CharToOrderMap, Constants.CHARSET_IBM866);
50 }
51
52
53 ////////////////////////////////////////////////////////////////
54 // constants
55 ////////////////////////////////////////////////////////////////
56 private static final short[] ibm866CharToOrderMap = new short[] {
57 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
58 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
59 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
60 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
61 253,142,143,144,145,146,147,148,149,150,151,152, 74,153, 75,154, //40
62 155,156,157,158,159,160,161,162,163,164,165,253,253,253,253,253, //50
63 253, 71,172, 66,173, 65,174, 76,175, 64,176,177, 77, 72,178, 69, //60
64 67,179, 78, 73,180,181, 79,182,183,184,185,253,253,253,253,253, //70
65 37, 44, 33, 46, 41, 48, 56, 51, 42, 60, 36, 49, 38, 31, 34, 35,
66 45, 32, 40, 52, 53, 55, 58, 50, 57, 63, 70, 62, 61, 47, 59, 43,
67 3, 21, 10, 19, 13, 2, 24, 20, 4, 23, 11, 8, 12, 5, 1, 15,
68 191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,
69 207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,
70 223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,
71 9, 7, 6, 14, 39, 26, 28, 22, 25, 29, 54, 18, 17, 30, 27, 16,
72 239, 68,240,241,242,243,244,245,246,247,248,249,250,251,252,255,
73 };
74 }
+0
-75
src/org/mozilla/universalchardet/prober/sequence/Koi8rModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Koi8rModel extends CyrillicModel
43 {
44 ////////////////////////////////////////////////////////////////
45 // methods
46 ////////////////////////////////////////////////////////////////
47 public Koi8rModel()
48 {
49 super(koi8rCharToOrderMap, Constants.CHARSET_KOI8_R);
50 }
51
52
53 ////////////////////////////////////////////////////////////////
54 // constants
55 ////////////////////////////////////////////////////////////////
56 private static final short[] koi8rCharToOrderMap = new short[] {
57 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
58 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
59 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
60 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
61 253,142,143,144,145,146,147,148,149,150,151,152, 74,153, 75,154, //40
62 155,156,157,158,159,160,161,162,163,164,165,253,253,253,253,253, //50
63 253, 71,172, 66,173, 65,174, 76,175, 64,176,177, 77, 72,178, 69, //60
64 67,179, 78, 73,180,181, 79,182,183,184,185,253,253,253,253,253, //70
65 191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206, //80
66 207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222, //90
67 223,224,225, 68,226,227,228,229,230,231,232,233,234,235,236,237, //a0
68 238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253, //b0
69 27, 3, 21, 28, 13, 2, 39, 19, 26, 4, 23, 11, 8, 12, 5, 1, //c0
70 15, 16, 9, 7, 6, 14, 24, 10, 17, 18, 20, 25, 30, 29, 22, 54, //d0
71 59, 37, 44, 58, 41, 48, 53, 46, 55, 42, 60, 36, 49, 38, 31, 34, //e0
72 35, 43, 45, 32, 40, 52, 56, 33, 61, 62, 51, 57, 47, 63, 50, 70, //f0
73 };
74 }
+0
-75
src/org/mozilla/universalchardet/prober/sequence/Latin5BulgarianModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Latin5BulgarianModel extends BulgarianModel
43 {
44 ////////////////////////////////////////////////////////////////
45 // methods
46 ////////////////////////////////////////////////////////////////
47 public Latin5BulgarianModel()
48 {
49 super(latin5BulgarianCharToOrderMap, Constants.CHARSET_ISO_8859_5);
50 }
51
52
53 ////////////////////////////////////////////////////////////////
54 // constants
55 ////////////////////////////////////////////////////////////////
56 private static final short[] latin5BulgarianCharToOrderMap = new short[] {
57 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
58 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
59 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
60 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
61 253, 77, 90, 99,100, 72,109,107,101, 79,185, 81,102, 76, 94, 82, //40
62 110,186,108, 91, 74,119, 84, 96,111,187,115,253,253,253,253,253, //50
63 253, 65, 69, 70, 66, 63, 68,112,103, 92,194,104, 95, 86, 87, 71, //60
64 116,195, 85, 93, 97,113,196,197,198,199,200,253,253,253,253,253, //70
65 194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209, //80
66 210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225, //90
67 81,226,227,228,229,230,105,231,232,233,234,235,236, 45,237,238, //a0
68 31, 32, 35, 43, 37, 44, 55, 47, 40, 59, 33, 46, 38, 36, 41, 30, //b0
69 39, 28, 34, 51, 48, 49, 53, 50, 54, 57, 61,239, 67,240, 60, 56, //c0
70 1, 18, 9, 20, 11, 3, 23, 15, 2, 26, 12, 10, 14, 6, 4, 13, //d0
71 7, 8, 5, 19, 29, 25, 22, 21, 27, 24, 17, 75, 52,241, 42, 16, //e0
72 62,242,243,244, 58,245, 98,246,247,248,249,250,251, 91,252,253, //f0
73 };
74 }
+0
-75
src/org/mozilla/universalchardet/prober/sequence/Latin5Model.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Latin5Model extends CyrillicModel
43 {
44 ////////////////////////////////////////////////////////////////
45 // methods
46 ////////////////////////////////////////////////////////////////
47 public Latin5Model()
48 {
49 super(latin5CharToOrderMap, Constants.CHARSET_ISO_8859_5);
50 }
51
52
53 ////////////////////////////////////////////////////////////////
54 // constants
55 ////////////////////////////////////////////////////////////////
56 private static final short[] latin5CharToOrderMap = new short[] {
57 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
58 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
59 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
60 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
61 253,142,143,144,145,146,147,148,149,150,151,152, 74,153, 75,154, //40
62 155,156,157,158,159,160,161,162,163,164,165,253,253,253,253,253, //50
63 253, 71,172, 66,173, 65,174, 76,175, 64,176,177, 77, 72,178, 69, //60
64 67,179, 78, 73,180,181, 79,182,183,184,185,253,253,253,253,253, //70
65 191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,
66 207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,
67 223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,
68 37, 44, 33, 46, 41, 48, 56, 51, 42, 60, 36, 49, 38, 31, 34, 35,
69 45, 32, 40, 52, 53, 55, 58, 50, 57, 63, 70, 62, 61, 47, 59, 43,
70 3, 21, 10, 19, 13, 2, 24, 20, 4, 23, 11, 8, 12, 5, 1, 15,
71 9, 7, 6, 14, 39, 26, 28, 22, 25, 29, 54, 18, 17, 30, 27, 16,
72 239, 68,240,241,242,243,244,245,246,247,248,249,250,251,252,255,
73 };
74 }
+0
-75
src/org/mozilla/universalchardet/prober/sequence/Latin7Model.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Latin7Model extends GreekModel
43 {
44 ////////////////////////////////////////////////////////////////
45 // methods
46 ////////////////////////////////////////////////////////////////
47 public Latin7Model()
48 {
49 super(latin7CharToOrderMap, Constants.CHARSET_ISO_8859_7);
50 }
51
52
53 ////////////////////////////////////////////////////////////////
54 // constants
55 ////////////////////////////////////////////////////////////////
56 private static final short[] latin7CharToOrderMap = new short[] {
57 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
58 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
59 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
60 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
61 253, 82,100,104, 94, 98,101,116,102,111,187,117, 92, 88,113, 85, //40
62 79,118,105, 83, 67,114,119, 95, 99,109,188,253,253,253,253,253, //50
63 253, 72, 70, 80, 81, 60, 96, 93, 89, 68,120, 97, 77, 86, 69, 55, //60
64 78,115, 65, 66, 58, 76,106,103, 87,107,112,253,253,253,253,253, //70
65 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //80
66 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //90
67 +253,233, 90,253,253,253,253,253,253,253,253,253,253, 74,253,253, //a0
68 253,253,253,253,247,248, 61, 36, 46, 71, 73,253, 54,253,108,123, //b0
69 110, 31, 51, 43, 41, 34, 91, 40, 52, 47, 44, 53, 38, 49, 59, 39, //c0
70 35, 48,250, 37, 33, 45, 56, 50, 84, 57,120,121, 17, 18, 22, 15, //d0
71 124, 1, 29, 20, 21, 3, 32, 13, 25, 5, 11, 16, 10, 6, 30, 4, //e0
72 9, 8, 14, 7, 2, 12, 28, 23, 42, 24, 64, 75, 19, 26, 27,253, //f0
73 };
74 }
+0
-75
src/org/mozilla/universalchardet/prober/sequence/MacCyrillicModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class MacCyrillicModel extends CyrillicModel
43 {
44 ////////////////////////////////////////////////////////////////
45 // methods
46 ////////////////////////////////////////////////////////////////
47 public MacCyrillicModel()
48 {
49 super(macCyrillicCharToOrderMap, Constants.CHARSET_MACCYRILLIC);
50 }
51
52
53 ////////////////////////////////////////////////////////////////
54 // constants
55 ////////////////////////////////////////////////////////////////
56 private static final short[] macCyrillicCharToOrderMap = new short[] {
57 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
58 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
59 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
60 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
61 253,142,143,144,145,146,147,148,149,150,151,152, 74,153, 75,154, //40
62 155,156,157,158,159,160,161,162,163,164,165,253,253,253,253,253, //50
63 253, 71,172, 66,173, 65,174, 76,175, 64,176,177, 77, 72,178, 69, //60
64 67,179, 78, 73,180,181, 79,182,183,184,185,253,253,253,253,253, //70
65 37, 44, 33, 46, 41, 48, 56, 51, 42, 60, 36, 49, 38, 31, 34, 35,
66 45, 32, 40, 52, 53, 55, 58, 50, 57, 63, 70, 62, 61, 47, 59, 43,
67 191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,
68 207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,
69 223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,
70 239,240,241,242,243,244,245,246,247,248,249,250,251,252, 68, 16,
71 3, 21, 10, 19, 13, 2, 24, 20, 4, 23, 11, 8, 12, 5, 1, 15,
72 9, 7, 6, 14, 39, 26, 28, 22, 25, 29, 54, 18, 17, 30, 27,255,
73 };
74 }
+0
-95
src/org/mozilla/universalchardet/prober/sequence/SequenceModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Universal charset detector code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 2001
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Shy Shalom <shooshX@gmail.com>
22 * Kohei TAKETA <k-taka@void.in> (Java port)
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober.sequence;
39
40 public abstract class SequenceModel
41 {
42 ////////////////////////////////////////////////////////////////
43 // fields
44 ////////////////////////////////////////////////////////////////
45 protected short[] charToOrderMap;
46 protected byte[] precedenceMatrix;
47 protected float typicalPositiveRatio;
48 protected boolean keepEnglishLetter;
49 protected String charsetName;
50
51
52 ////////////////////////////////////////////////////////////////
53 // methods
54 ////////////////////////////////////////////////////////////////
55 public SequenceModel(
56 short[] charToOrderMap,
57 byte[] precedenceMatrix,
58 float typicalPositiveRatio,
59 boolean keepEnglishLetter,
60 String charsetName)
61 {
62 this.charToOrderMap = charToOrderMap;
63 this.precedenceMatrix = precedenceMatrix;
64 this.typicalPositiveRatio = typicalPositiveRatio;
65 this.keepEnglishLetter = keepEnglishLetter;
66 this.charsetName = charsetName;
67 }
68
69 public short getOrder(byte b)
70 {
71 int c = b & 0xFF;
72 return this.charToOrderMap[c];
73 }
74
75 public byte getPrecedence(int pos)
76 {
77 return this.precedenceMatrix[pos];
78 }
79
80 public float getTypicalPositiveRatio()
81 {
82 return this.typicalPositiveRatio;
83 }
84
85 public boolean getKeepEnglishLetter()
86 {
87 return this.keepEnglishLetter;
88 }
89
90 public String getCharsetName()
91 {
92 return this.charsetName;
93 }
94 }
+0
-75
src/org/mozilla/universalchardet/prober/sequence/Win1251BulgarianModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Win1251BulgarianModel extends BulgarianModel
43 {
44 ////////////////////////////////////////////////////////////////
45 // methods
46 ////////////////////////////////////////////////////////////////
47 public Win1251BulgarianModel()
48 {
49 super(win1251BulgarianCharToOrderMap, Constants.CHARSET_WINDOWS_1251);
50 }
51
52
53 ////////////////////////////////////////////////////////////////
54 // constants
55 ////////////////////////////////////////////////////////////////
56 private static final short[] win1251BulgarianCharToOrderMap = new short[] {
57 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
58 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
59 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
60 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
61 253, 77, 90, 99,100, 72,109,107,101, 79,185, 81,102, 76, 94, 82, //40
62 110,186,108, 91, 74,119, 84, 96,111,187,115,253,253,253,253,253, //50
63 253, 65, 69, 70, 66, 63, 68,112,103, 92,194,104, 95, 86, 87, 71, //60
64 116,195, 85, 93, 97,113,196,197,198,199,200,253,253,253,253,253, //70
65 206,207,208,209,210,211,212,213,120,214,215,216,217,218,219,220, //80
66 221, 78, 64, 83,121, 98,117,105,222,223,224,225,226,227,228,229, //90
67 88,230,231,232,233,122, 89,106,234,235,236,237,238, 45,239,240, //a0
68 73, 80,118,114,241,242,243,244,245, 62, 58,246,247,248,249,250, //b0
69 31, 32, 35, 43, 37, 44, 55, 47, 40, 59, 33, 46, 38, 36, 41, 30, //c0
70 39, 28, 34, 51, 48, 49, 53, 50, 54, 57, 61,251, 67,252, 60, 56, //d0
71 1, 18, 9, 20, 11, 3, 23, 15, 2, 26, 12, 10, 14, 6, 4, 13, //e0
72 7, 8, 5, 19, 29, 25, 22, 21, 27, 24, 17, 75, 52,253, 42, 16, //f0
73 };
74 }
+0
-75
src/org/mozilla/universalchardet/prober/sequence/Win1251Model.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.sequence;
38
39 import org.mozilla.universalchardet.Constants;
40
41
42 public class Win1251Model extends CyrillicModel
43 {
44 ////////////////////////////////////////////////////////////////
45 // methods
46 ////////////////////////////////////////////////////////////////
47 public Win1251Model()
48 {
49 super(win1251CharToOrderMap, Constants.CHARSET_WINDOWS_1251);
50 }
51
52
53 ////////////////////////////////////////////////////////////////
54 // constants
55 ////////////////////////////////////////////////////////////////
56 private static short[] win1251CharToOrderMap = new short[] {
57 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
58 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
59 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
60 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
61 253,142,143,144,145,146,147,148,149,150,151,152, 74,153, 75,154, //40
62 155,156,157,158,159,160,161,162,163,164,165,253,253,253,253,253, //50
63 253, 71,172, 66,173, 65,174, 76,175, 64,176,177, 77, 72,178, 69, //60
64 67,179, 78, 73,180,181, 79,182,183,184,185,253,253,253,253,253, //70
65 191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,
66 207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,
67 223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,
68 239,240,241,242,243,244,245,246, 68,247,248,249,250,251,252,253,
69 37, 44, 33, 46, 41, 48, 56, 51, 42, 60, 36, 49, 38, 31, 34, 35,
70 45, 32, 40, 52, 53, 55, 58, 50, 57, 63, 70, 62, 61, 47, 59, 43,
71 3, 21, 10, 19, 13, 2, 24, 20, 4, 23, 11, 8, 12, 5, 1, 15,
72 9, 7, 6, 14, 39, 26, 28, 22, 25, 29, 54, 18, 17, 30, 27, 16,
73 };
74 }
+0
-74
src/org/mozilla/universalchardet/prober/sequence/Win1253Model.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is Mozilla Communicator client code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36 package org.mozilla.universalchardet.prober.sequence;
37
38 import org.mozilla.universalchardet.Constants;
39
40
41 public class Win1253Model extends GreekModel
42 {
43 ////////////////////////////////////////////////////////////////
44 // methods
45 ////////////////////////////////////////////////////////////////
46 public Win1253Model()
47 {
48 super(win1253CharToOrderMap, Constants.CHARSET_WINDOWS_1253);
49 }
50
51
52 ////////////////////////////////////////////////////////////////
53 // constants
54 ////////////////////////////////////////////////////////////////
55 private static final short[] win1253CharToOrderMap = new short[] {
56 255,255,255,255,255,255,255,255,255,255,254,255,255,254,255,255, //00
57 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //10
58 253,253,253,253,253,253,253,253,253,253,253,253,253,253,253,253, //20
59 252,252,252,252,252,252,252,252,252,252,253,253,253,253,253,253, //30
60 253, 82,100,104, 94, 98,101,116,102,111,187,117, 92, 88,113, 85, //40
61 79,118,105, 83, 67,114,119, 95, 99,109,188,253,253,253,253,253, //50
62 253, 72, 70, 80, 81, 60, 96, 93, 89, 68,120, 97, 77, 86, 69, 55, //60
63 78,115, 65, 66, 58, 76,106,103, 87,107,112,253,253,253,253,253, //70
64 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //80
65 255,255,255,255,255,255,255,255,255,255,255,255,255,255,255,255, //90
66 253,233, 61,253,253,253,253,253,253,253,253,253,253, 74,253,253, //a0
67 253,253,253,253,247,253,253, 36, 46, 71, 73,253, 54,253,108,123, //b0
68 110, 31, 51, 43, 41, 34, 91, 40, 52, 47, 44, 53, 38, 49, 59, 39, //c0
69 35, 48,250, 37, 33, 45, 56, 50, 84, 57,120,121, 17, 18, 22, 15, //d0
70 124, 1, 29, 20, 21, 3, 32, 13, 25, 5, 11, 16, 10, 6, 30, 4, //e0
71 9, 8, 14, 7, 2, 12, 28, 23, 42, 24, 64, 75, 19, 26, 27,253, //f0
72 };
73 }
+0
-118
src/org/mozilla/universalchardet/prober/statemachine/Big5SMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class Big5SMModel extends SMModel
47 {
48 ////////////////////////////////////////////////////////////////
49 // constants
50 ////////////////////////////////////////////////////////////////
51 public static final int BIG5_CLASS_FACTOR = 5;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public Big5SMModel()
58 {
59 super(
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, big5ClassTable),
61 BIG5_CLASS_FACTOR,
62 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, big5StateTable),
63 big5CharLenTable,
64 Constants.CHARSET_BIG5
65 );
66 }
67
68
69 ////////////////////////////////////////////////////////////////
70 // constants continued
71 ////////////////////////////////////////////////////////////////
72 private static int[] big5ClassTable = new int[] {
73 // PkgInt.pack4bits(0,1,1,1,1,1,1,1), // 00 - 07
74 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 00 - 07 //allow 0x00 as legal value
75 PkgInt.pack4bits(1,1,1,1,1,1,0,0), // 08 - 0f
76 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 10 - 17
77 PkgInt.pack4bits(1,1,1,0,1,1,1,1), // 18 - 1f
78 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 20 - 27
79 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 28 - 2f
80 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 30 - 37
81 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 38 - 3f
82 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 40 - 47
83 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 48 - 4f
84 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 50 - 57
85 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 58 - 5f
86 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 60 - 67
87 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 68 - 6f
88 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 70 - 77
89 PkgInt.pack4bits(2,2,2,2,2,2,2,1), // 78 - 7f
90 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 80 - 87
91 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 88 - 8f
92 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 90 - 97
93 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 98 - 9f
94 PkgInt.pack4bits(4,3,3,3,3,3,3,3), // a0 - a7
95 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // a8 - af
96 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // b0 - b7
97 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // b8 - bf
98 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // c0 - c7
99 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // c8 - cf
100 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // d0 - d7
101 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // d8 - df
102 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // e0 - e7
103 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // e8 - ef
104 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // f0 - f7
105 PkgInt.pack4bits(3,3,3,3,3,3,3,0) // f8 - ff
106 };
107
108 private static int[] big5StateTable = new int[] {
109 PkgInt.pack4bits(ERROR,START,START, 3,ERROR,ERROR,ERROR,ERROR),//00-07
110 PkgInt.pack4bits(ERROR,ERROR,ITSME,ITSME,ITSME,ITSME,ITSME,ERROR),//08-0f
111 PkgInt.pack4bits(ERROR,START,START,START,START,START,START,START) //10-17
112 };
113
114 private static int[] big5CharLenTable = new int[] {
115 0, 1, 1, 2, 0
116 };
117 }
+0
-81
src/org/mozilla/universalchardet/prober/statemachine/CodingStateMachine.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 public class CodingStateMachine
40 {
41 protected SMModel model;
42 protected int currentState;
43 protected int currentCharLen;
44 protected int currentBytePos;
45
46 public CodingStateMachine(SMModel model)
47 {
48 this.model = model;
49 this.currentState = SMModel.START;
50 }
51
52 public int nextState(byte c)
53 {
54 int byteCls = this.model.getClass(c);
55 if (this.currentState == SMModel.START) {
56 this.currentBytePos = 0;
57 this.currentCharLen = this.model.getCharLen(byteCls);
58 }
59
60 this.currentState = this.model.getNextState(byteCls, this.currentState);
61 ++this.currentBytePos;
62
63 return this.currentState;
64 }
65
66 public int getCurrentCharLen()
67 {
68 return this.currentCharLen;
69 }
70
71 public void reset()
72 {
73 this.currentState = SMModel.START;
74 }
75
76 public String getCodingStateMachine()
77 {
78 return this.model.getName();
79 }
80 }
+0
-120
src/org/mozilla/universalchardet/prober/statemachine/EUCJPSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class EUCJPSMModel extends SMModel
47 {
48 ////////////////////////////////////////////////////////////////
49 // constants
50 ////////////////////////////////////////////////////////////////
51 public static final int EUCJP_CLASS_FACTOR = 6;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public EUCJPSMModel()
58 {
59 super(
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, eucjpClassTable),
61 EUCJP_CLASS_FACTOR,
62 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, eucjpStateTable),
63 eucjpCharLenTable,
64 Constants.CHARSET_EUC_JP
65 );
66 }
67
68
69 ////////////////////////////////////////////////////////////////
70 // constants continued
71 ////////////////////////////////////////////////////////////////
72 private static int[] eucjpClassTable = new int[] {
73 // PkgInt.pack4bits(5,4,4,4,4,4,4,4), // 00 - 07
74 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 00 - 07
75 PkgInt.pack4bits(4,4,4,4,4,4,5,5), // 08 - 0f
76 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 10 - 17
77 PkgInt.pack4bits(4,4,4,5,4,4,4,4), // 18 - 1f
78 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 20 - 27
79 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 28 - 2f
80 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 30 - 37
81 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 38 - 3f
82 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 40 - 47
83 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 48 - 4f
84 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 50 - 57
85 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 58 - 5f
86 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 60 - 67
87 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 68 - 6f
88 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 70 - 77
89 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 78 - 7f
90 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // 80 - 87
91 PkgInt.pack4bits(5,5,5,5,5,5,1,3), // 88 - 8f
92 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // 90 - 97
93 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // 98 - 9f
94 PkgInt.pack4bits(5,2,2,2,2,2,2,2), // a0 - a7
95 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a8 - af
96 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b0 - b7
97 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b8 - bf
98 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c0 - c7
99 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c8 - cf
100 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d0 - d7
101 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d8 - df
102 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // e0 - e7
103 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // e8 - ef
104 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // f0 - f7
105 PkgInt.pack4bits(0,0,0,0,0,0,0,5) // f8 - ff
106 };
107
108 private static int[] eucjpStateTable = new int[] {
109 PkgInt.pack4bits( 3, 4, 3, 5,START,ERROR,ERROR,ERROR),//00-07
110 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ITSME,ITSME,ITSME),//08-0f
111 PkgInt.pack4bits(ITSME,ITSME,START,ERROR,START,ERROR,ERROR,ERROR),//10-17
112 PkgInt.pack4bits(ERROR,ERROR,START,ERROR,ERROR,ERROR, 3,ERROR),//18-1f
113 PkgInt.pack4bits( 3,ERROR,ERROR,ERROR,START,START,START,START) //20-27
114 };
115
116 private static int[] eucjpCharLenTable = new int[] {
117 2, 2, 2, 3, 1, 0
118 };
119 }
+0
-118
src/org/mozilla/universalchardet/prober/statemachine/EUCKRSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class EUCKRSMModel extends SMModel
47 {
48 ////////////////////////////////////////////////////////////////
49 // constants
50 ////////////////////////////////////////////////////////////////
51 public static final int EUCKR_CLASS_FACTOR = 4;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public EUCKRSMModel()
58 {
59 super(
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, euckrClassTable),
61 EUCKR_CLASS_FACTOR,
62 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, euckrStateTable),
63 euckrCharLenTable,
64 Constants.CHARSET_EUC_KR
65 );
66 }
67
68
69 ////////////////////////////////////////////////////////////////
70 // constants continued
71 ////////////////////////////////////////////////////////////////
72 private static int[] euckrClassTable = new int[] {
73 // PkgInt.pack4bits(0,1,1,1,1,1,1,1), // 00 - 07
74 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 00 - 07
75 PkgInt.pack4bits(1,1,1,1,1,1,0,0), // 08 - 0f
76 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 10 - 17
77 PkgInt.pack4bits(1,1,1,0,1,1,1,1), // 18 - 1f
78 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 20 - 27
79 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 28 - 2f
80 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 30 - 37
81 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 38 - 3f
82 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 40 - 47
83 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 48 - 4f
84 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 50 - 57
85 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 58 - 5f
86 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 60 - 67
87 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 68 - 6f
88 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 70 - 77
89 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 78 - 7f
90 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 80 - 87
91 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 88 - 8f
92 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 90 - 97
93 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 98 - 9f
94 PkgInt.pack4bits(0,2,2,2,2,2,2,2), // a0 - a7
95 PkgInt.pack4bits(2,2,2,2,2,3,3,3), // a8 - af
96 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b0 - b7
97 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b8 - bf
98 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c0 - c7
99 PkgInt.pack4bits(2,3,2,2,2,2,2,2), // c8 - cf
100 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d0 - d7
101 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d8 - df
102 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e0 - e7
103 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e8 - ef
104 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // f0 - f7
105 PkgInt.pack4bits(2,2,2,2,2,2,2,0) // f8 - ff
106 };
107
108 private static int[] euckrStateTable = new int[] {
109 PkgInt.pack4bits(ERROR,START, 3,ERROR,ERROR,ERROR,ERROR,ERROR),//00-07
110 PkgInt.pack4bits(ITSME,ITSME,ITSME,ITSME,ERROR,ERROR,START,START) //08-0f
111 };
112
113 private static int[] euckrCharLenTable = new int[] {
114 0, 1, 2, 0
115 };
116
117 }
+0
-121
src/org/mozilla/universalchardet/prober/statemachine/EUCTWSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class EUCTWSMModel extends SMModel
47 {
48 ////////////////////////////////////////////////////////////////
49 // constants
50 ////////////////////////////////////////////////////////////////
51 public static final int EUCTW_CLASS_FACTOR = 7;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public EUCTWSMModel()
58 {
59 super(
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, euctwClassTable),
61 EUCTW_CLASS_FACTOR,
62 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, euctwStateTable),
63 euctwCharLenTable,
64 Constants.CHARSET_EUC_TW
65 );
66 }
67
68
69 ////////////////////////////////////////////////////////////////
70 // constants continued
71 ////////////////////////////////////////////////////////////////
72 private static int[] euctwClassTable = new int[] {
73 // PkgInt.pack4bits(0,2,2,2,2,2,2,2), // 00 - 07
74 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 00 - 07
75 PkgInt.pack4bits(2,2,2,2,2,2,0,0), // 08 - 0f
76 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 10 - 17
77 PkgInt.pack4bits(2,2,2,0,2,2,2,2), // 18 - 1f
78 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 20 - 27
79 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 28 - 2f
80 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 30 - 37
81 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 38 - 3f
82 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 40 - 47
83 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 48 - 4f
84 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 50 - 57
85 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 58 - 5f
86 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 60 - 67
87 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 68 - 6f
88 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 70 - 77
89 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 78 - 7f
90 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 80 - 87
91 PkgInt.pack4bits(0,0,0,0,0,0,6,0), // 88 - 8f
92 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 90 - 97
93 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 98 - 9f
94 PkgInt.pack4bits(0,3,4,4,4,4,4,4), // a0 - a7
95 PkgInt.pack4bits(5,5,1,1,1,1,1,1), // a8 - af
96 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // b0 - b7
97 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // b8 - bf
98 PkgInt.pack4bits(1,1,3,1,3,3,3,3), // c0 - c7
99 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // c8 - cf
100 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // d0 - d7
101 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // d8 - df
102 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // e0 - e7
103 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // e8 - ef
104 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // f0 - f7
105 PkgInt.pack4bits(3,3,3,3,3,3,3,0) // f8 - ff
106 };
107
108 private static int[] euctwStateTable = new int[] {
109 PkgInt.pack4bits(ERROR,ERROR,START, 3, 3, 3, 4,ERROR),//00-07
110 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ITSME,ITSME),//08-0f
111 PkgInt.pack4bits(ITSME,ITSME,ITSME,ITSME,ITSME,ERROR,START,ERROR),//10-17
112 PkgInt.pack4bits(START,START,START,ERROR,ERROR,ERROR,ERROR,ERROR),//18-1f
113 PkgInt.pack4bits( 5,ERROR,ERROR,ERROR,START,ERROR,START,START),//20-27
114 PkgInt.pack4bits(START,ERROR,START,START,START,START,START,START) //28-2f
115 };
116
117 private static int[] euctwCharLenTable = new int[] {
118 0, 0, 1, 2, 2, 2, 3
119 };
120 }
+0
-120
src/org/mozilla/universalchardet/prober/statemachine/GB18030SMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class GB18030SMModel extends SMModel
47 {
48 ////////////////////////////////////////////////////////////////
49 // constants
50 ////////////////////////////////////////////////////////////////
51 public static final int GB18030_CLASS_FACTOR = 7;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public GB18030SMModel()
58 {
59 super(
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, gb18030ClassTable),
61 GB18030_CLASS_FACTOR,
62 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, gb18030StateTable),
63 gb18030CharLenTable,
64 Constants.CHARSET_GB18030
65 );
66 }
67
68
69 ////////////////////////////////////////////////////////////////
70 // constants continued
71 ////////////////////////////////////////////////////////////////
72 private static int[] gb18030ClassTable = new int[] {
73 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 00 - 07
74 PkgInt.pack4bits(1,1,1,1,1,1,0,0), // 08 - 0f
75 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 10 - 17
76 PkgInt.pack4bits(1,1,1,0,1,1,1,1), // 18 - 1f
77 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 20 - 27
78 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 28 - 2f
79 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // 30 - 37
80 PkgInt.pack4bits(3,3,1,1,1,1,1,1), // 38 - 3f
81 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 40 - 47
82 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 48 - 4f
83 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 50 - 57
84 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 58 - 5f
85 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 60 - 67
86 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 68 - 6f
87 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 70 - 77
88 PkgInt.pack4bits(2,2,2,2,2,2,2,4), // 78 - 7f
89 PkgInt.pack4bits(5,6,6,6,6,6,6,6), // 80 - 87
90 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // 88 - 8f
91 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // 90 - 97
92 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // 98 - 9f
93 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // a0 - a7
94 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // a8 - af
95 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // b0 - b7
96 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // b8 - bf
97 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // c0 - c7
98 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // c8 - cf
99 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // d0 - d7
100 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // d8 - df
101 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // e0 - e7
102 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // e8 - ef
103 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // f0 - f7
104 PkgInt.pack4bits(6,6,6,6,6,6,6,0) // f8 - ff
105 };
106
107 private static int[] gb18030StateTable = new int[] {
108 PkgInt.pack4bits(ERROR,START,START,START,START,START, 3,ERROR),//00-07
109 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ITSME,ITSME),//08-0f
110 PkgInt.pack4bits(ITSME,ITSME,ITSME,ITSME,ITSME,ERROR,ERROR,START),//10-17
111 PkgInt.pack4bits( 4,ERROR,START,START,ERROR,ERROR,ERROR,ERROR),//18-1f
112 PkgInt.pack4bits(ERROR,ERROR, 5,ERROR,ERROR,ERROR,ITSME,ERROR),//20-27
113 PkgInt.pack4bits(ERROR,ERROR,START,START,START,START,START,START) //28-2f
114 };
115
116 private static int[] gb18030CharLenTable = new int[] {
117 0, 1, 1, 1, 1, 1, 2
118 };
119 }
+0
-120
src/org/mozilla/universalchardet/prober/statemachine/HZSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class HZSMModel extends SMModel
47 {
48 ////////////////////////////////////////////////////////////////
49 // constants
50 ////////////////////////////////////////////////////////////////
51 public static final int HZS_CLASS_FACTOR = 6;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public HZSMModel()
58 {
59 super(
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, hzsClassTable),
61 HZS_CLASS_FACTOR,
62 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, hzsStateTable),
63 hzsCharLenTable,
64 Constants.CHARSET_HZ_GB_2312
65 );
66 }
67
68
69 ////////////////////////////////////////////////////////////////
70 // constants continued
71 ////////////////////////////////////////////////////////////////
72 private static int[] hzsClassTable = new int[] {
73 PkgInt.pack4bits(1,0,0,0,0,0,0,0), // 00 - 07
74 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 08 - 0f
75 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 10 - 17
76 PkgInt.pack4bits(0,0,0,1,0,0,0,0), // 18 - 1f
77 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 20 - 27
78 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 28 - 2f
79 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 30 - 37
80 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 38 - 3f
81 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 40 - 47
82 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 48 - 4f
83 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 50 - 57
84 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 58 - 5f
85 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 60 - 67
86 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 68 - 6f
87 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 70 - 77
88 PkgInt.pack4bits(0,0,0,4,0,5,2,0), // 78 - 7f
89 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 80 - 87
90 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 88 - 8f
91 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 90 - 97
92 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 98 - 9f
93 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // a0 - a7
94 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // a8 - af
95 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // b0 - b7
96 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // b8 - bf
97 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // c0 - c7
98 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // c8 - cf
99 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // d0 - d7
100 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // d8 - df
101 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // e0 - e7
102 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // e8 - ef
103 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // f0 - f7
104 PkgInt.pack4bits(1,1,1,1,1,1,1,1) // f8 - ff
105 };
106
107 private static int[] hzsStateTable = new int[] {
108 PkgInt.pack4bits(START,ERROR, 3,START,START,START,ERROR,ERROR),//00-07
109 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ITSME,ITSME,ITSME),//08-0f
110 PkgInt.pack4bits(ITSME,ITSME,ERROR,ERROR,START,START, 4,ERROR),//10-17
111 PkgInt.pack4bits( 5,ERROR, 6,ERROR, 5, 5, 4,ERROR),//18-1f
112 PkgInt.pack4bits( 4,ERROR, 4, 4, 4,ERROR, 4,ERROR),//20-27
113 PkgInt.pack4bits( 4,ITSME,START,START,START,START,START,START) //28-2f
114 };
115
116 private static int[] hzsCharLenTable = new int[] {
117 0, 0, 0, 0, 0, 0
118 };
119 }
+0
-122
src/org/mozilla/universalchardet/prober/statemachine/ISO2022CNSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class ISO2022CNSMModel extends SMModel
47 {
48 ////////////////////////////////////////////////////////////////
49 // constants
50 ////////////////////////////////////////////////////////////////
51 public static final int ISO2022CN_CLASS_FACTOR = 9;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public ISO2022CNSMModel()
58 {
59 super(
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, iso2022cnClassTable),
61 ISO2022CN_CLASS_FACTOR,
62 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, iso2022cnStateTable),
63 iso2022cnCharLenTable,
64 Constants.CHARSET_ISO_2022_CN
65 );
66 }
67
68
69 ////////////////////////////////////////////////////////////////
70 // constants continued
71 ////////////////////////////////////////////////////////////////
72 private static int[] iso2022cnClassTable = new int[] {
73 PkgInt.pack4bits(2,0,0,0,0,0,0,0), // 00 - 07
74 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 08 - 0f
75 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 10 - 17
76 PkgInt.pack4bits(0,0,0,1,0,0,0,0), // 18 - 1f
77 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 20 - 27
78 PkgInt.pack4bits(0,3,0,0,0,0,0,0), // 28 - 2f
79 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 30 - 37
80 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 38 - 3f
81 PkgInt.pack4bits(0,0,0,4,0,0,0,0), // 40 - 47
82 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 48 - 4f
83 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 50 - 57
84 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 58 - 5f
85 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 60 - 67
86 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 68 - 6f
87 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 70 - 77
88 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 78 - 7f
89 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 80 - 87
90 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 88 - 8f
91 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 90 - 97
92 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 98 - 9f
93 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a0 - a7
94 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a8 - af
95 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b0 - b7
96 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b8 - bf
97 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c0 - c7
98 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c8 - cf
99 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d0 - d7
100 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d8 - df
101 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e0 - e7
102 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e8 - ef
103 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // f0 - f7
104 PkgInt.pack4bits(2,2,2,2,2,2,2,2) // f8 - ff
105 };
106
107 private static int[] iso2022cnStateTable = new int[] {
108 PkgInt.pack4bits(START, 3,ERROR,START,START,START,START,START),//00-07
109 PkgInt.pack4bits(START,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//08-0f
110 PkgInt.pack4bits(ERROR,ERROR,ITSME,ITSME,ITSME,ITSME,ITSME,ITSME),//10-17
111 PkgInt.pack4bits(ITSME,ITSME,ITSME,ERROR,ERROR,ERROR, 4,ERROR),//18-1f
112 PkgInt.pack4bits(ERROR,ERROR,ERROR,ITSME,ERROR,ERROR,ERROR,ERROR),//20-27
113 PkgInt.pack4bits( 5, 6,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//28-2f
114 PkgInt.pack4bits(ERROR,ERROR,ERROR,ITSME,ERROR,ERROR,ERROR,ERROR),//30-37
115 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ITSME,ERROR,START) //38-3f
116 };
117
118 private static int[] iso2022cnCharLenTable = new int[] {
119 0, 0, 0, 0, 0, 0, 0, 0, 0
120 };
121 }
+0
-124
src/org/mozilla/universalchardet/prober/statemachine/ISO2022JPSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 * Kazutoshi Satoda
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37
38 package org.mozilla.universalchardet.prober.statemachine;
39
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
43 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
44 import org.mozilla.universalchardet.Constants;
45
46
47 public class ISO2022JPSMModel extends SMModel
48 {
49 ////////////////////////////////////////////////////////////////
50 // constants
51 ////////////////////////////////////////////////////////////////
52 public static final int ISO2022JP_CLASS_FACTOR = 10;
53
54
55 ////////////////////////////////////////////////////////////////
56 // methods
57 ////////////////////////////////////////////////////////////////
58 public ISO2022JPSMModel()
59 {
60 super(
61 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, iso2022jpClassTable),
62 ISO2022JP_CLASS_FACTOR,
63 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, iso2022jpStateTable),
64 iso2022jpCharLenTable,
65 Constants.CHARSET_ISO_2022_JP
66 );
67 }
68
69
70 ////////////////////////////////////////////////////////////////
71 // constants continued
72 ////////////////////////////////////////////////////////////////
73 private static int[] iso2022jpClassTable = new int[] {
74 PkgInt.pack4bits(2,0,0,0,0,0,0,0), // 00 - 07
75 PkgInt.pack4bits(0,0,0,0,0,0,2,2), // 08 - 0f
76 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 10 - 17
77 PkgInt.pack4bits(0,0,0,1,0,0,0,0), // 18 - 1f
78 PkgInt.pack4bits(0,0,0,0,7,0,0,0), // 20 - 27
79 PkgInt.pack4bits(3,0,0,0,0,0,0,0), // 28 - 2f
80 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 30 - 37
81 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 38 - 3f
82 PkgInt.pack4bits(6,0,4,0,8,0,0,0), // 40 - 47
83 PkgInt.pack4bits(0,9,5,0,0,0,0,0), // 48 - 4f
84 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 50 - 57
85 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 58 - 5f
86 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 60 - 67
87 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 68 - 6f
88 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 70 - 77
89 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 78 - 7f
90 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 80 - 87
91 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 88 - 8f
92 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 90 - 97
93 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 98 - 9f
94 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a0 - a7
95 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a8 - af
96 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b0 - b7
97 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b8 - bf
98 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c0 - c7
99 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c8 - cf
100 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d0 - d7
101 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d8 - df
102 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e0 - e7
103 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e8 - ef
104 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // f0 - f7
105 PkgInt.pack4bits(2,2,2,2,2,2,2,2) // f8 - ff
106 };
107
108 private static int[] iso2022jpStateTable = new int[] {
109 PkgInt.pack4bits(START, 3,ERROR,START,START,START,START,START),//00-07
110 PkgInt.pack4bits(START,START,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//08-0f
111 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ITSME,ITSME,ITSME),//10-17
112 PkgInt.pack4bits(ITSME,ITSME,ITSME,ITSME,ITSME,ITSME,ERROR,ERROR),//18-1f
113 PkgInt.pack4bits(ERROR, 5,ERROR,ERROR,ERROR, 4,ERROR,ERROR),//20-27
114 PkgInt.pack4bits(ERROR,ERROR,ERROR, 6,ITSME,ERROR,ITSME,ERROR),//28-2f
115 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ITSME,ITSME),//30-37
116 PkgInt.pack4bits(ERROR,ERROR,ERROR,ITSME,ERROR,ERROR,ERROR,ERROR),//38-3f
117 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ERROR,START,START) //40-47
118 };
119
120 private static int[] iso2022jpCharLenTable = new int[] {
121 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
122 };
123 }
+0
-119
src/org/mozilla/universalchardet/prober/statemachine/ISO2022KRSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class ISO2022KRSMModel extends SMModel
47 {
48 ////////////////////////////////////////////////////////////////
49 // constants
50 ////////////////////////////////////////////////////////////////
51 public static final int ISO2022KR_CLASS_FACTOR = 6;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public ISO2022KRSMModel()
58 {
59 super(
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, iso2022krClassTable),
61 ISO2022KR_CLASS_FACTOR,
62 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, iso2022krStateTable),
63 iso2022krCharLenTable,
64 Constants.CHARSET_ISO_2022_KR
65 );
66 }
67
68
69 ////////////////////////////////////////////////////////////////
70 // constants continued
71 ////////////////////////////////////////////////////////////////
72 private static int[] iso2022krClassTable = new int[] {
73 PkgInt.pack4bits(2,0,0,0,0,0,0,0), // 00 - 07
74 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 08 - 0f
75 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 10 - 17
76 PkgInt.pack4bits(0,0,0,1,0,0,0,0), // 18 - 1f
77 PkgInt.pack4bits(0,0,0,0,3,0,0,0), // 20 - 27
78 PkgInt.pack4bits(0,4,0,0,0,0,0,0), // 28 - 2f
79 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 30 - 37
80 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 38 - 3f
81 PkgInt.pack4bits(0,0,0,5,0,0,0,0), // 40 - 47
82 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 48 - 4f
83 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 50 - 57
84 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 58 - 5f
85 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 60 - 67
86 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 68 - 6f
87 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 70 - 77
88 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 78 - 7f
89 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 80 - 87
90 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 88 - 8f
91 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 90 - 97
92 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 98 - 9f
93 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a0 - a7
94 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a8 - af
95 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b0 - b7
96 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b8 - bf
97 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c0 - c7
98 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c8 - cf
99 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d0 - d7
100 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d8 - df
101 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e0 - e7
102 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // e8 - ef
103 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // f0 - f7
104 PkgInt.pack4bits(2,2,2,2,2,2,2,2) // f8 - ff
105 };
106
107 private static int[] iso2022krStateTable = new int[] {
108 PkgInt.pack4bits(START, 3,ERROR,START,START,START,ERROR,ERROR),//00-07
109 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ITSME,ITSME,ITSME),//08-0f
110 PkgInt.pack4bits(ITSME,ITSME,ERROR,ERROR,ERROR, 4,ERROR,ERROR),//10-17
111 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR, 5,ERROR,ERROR,ERROR),//18-1f
112 PkgInt.pack4bits(ERROR,ERROR,ERROR,ITSME,START,START,START,START) //20-27
113 };
114
115 private static int[] iso2022krCharLenTable = new int[] {
116 0, 0, 0, 0, 0, 0
117 };
118 }
+0
-118
src/org/mozilla/universalchardet/prober/statemachine/PkgInt.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 public class PkgInt
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final int INDEX_SHIFT_4BITS = 3;
45 public static final int INDEX_SHIFT_8BITS = 2;
46 public static final int INDEX_SHIFT_16BITS = 1;
47
48 public static final int SHIFT_MASK_4BITS = 7;
49 public static final int SHIFT_MASK_8BITS = 3;
50 public static final int SHIFT_MASK_16BITS = 1;
51
52 public static final int BIT_SHIFT_4BITS = 2;
53 public static final int BIT_SHIFT_8BITS = 3;
54 public static final int BIT_SHIFT_16BITS = 4;
55
56 public static final int UNIT_MASK_4BITS = 0x0000000F;
57 public static final int UNIT_MASK_8BITS = 0x000000FF;
58 public static final int UNIT_MASK_16BITS = 0x0000FFFF;
59
60
61 ////////////////////////////////////////////////////////////////
62 // fields
63 ////////////////////////////////////////////////////////////////
64 private int indexShift;
65 private int shiftMask;
66 private int bitShift;
67 private int unitMask;
68 private int[] data;
69
70
71 ////////////////////////////////////////////////////////////////
72 // methods
73 ////////////////////////////////////////////////////////////////
74 public PkgInt(
75 int indexShift,
76 int shiftMask,
77 int bitShift,
78 int unitMask,
79 int[] data)
80 {
81 this.indexShift = indexShift;
82 this.shiftMask = shiftMask;
83 this.bitShift = bitShift;
84 this.unitMask = unitMask;
85 this.data = data;
86 }
87
88 public static int pack16bits(int a, int b)
89 {
90 return ((b << 16) | a);
91 }
92
93 public static int pack8bits(int a, int b, int c, int d)
94 {
95 return pack16bits(
96 (b << 8) | a,
97 (d << 8) | c
98 );
99 }
100
101 public static int pack4bits(int a, int b, int c, int d, int e, int f, int g, int h)
102 {
103 return pack8bits(
104 (b << 4) | a,
105 (d << 4) | c,
106 (f << 4) | e,
107 (h << 4) | g
108 );
109 }
110
111 public int unpack(int i)
112 {
113 return (
114 (this.data[i>>this.indexShift] >> ((i&this.shiftMask)<<this.bitShift)) & this.unitMask
115 );
116 }
117 }
+0
-31
src/org/mozilla/universalchardet/prober/statemachine/PkgIntTest.java less more
0 package org.mozilla.universalchardet.prober.statemachine;
1
2 import static org.junit.Assert.*;
3
4 import org.junit.Test;
5
6 public class PkgIntTest
7 {
8 @Test
9 public void testUnpack()
10 {
11 int[] data = new int[] {
12 PkgInt.pack4bits(0, 1, 2, 3, 4, 5, 6, 7),
13 PkgInt.pack4bits(8, 9, 10, 11, 12, 13, 14, 15)
14 };
15
16 PkgInt pkg = new PkgInt(
17 PkgInt.INDEX_SHIFT_4BITS,
18 PkgInt.SHIFT_MASK_4BITS,
19 PkgInt.BIT_SHIFT_4BITS,
20 PkgInt.UNIT_MASK_4BITS,
21 data);
22
23 for (int i=0; i<16; ++i) {
24 int n = pkg.unpack(i);
25 if (n != i) {
26 fail("Invalid packed value.");
27 }
28 }
29 }
30 }
+0
-120
src/org/mozilla/universalchardet/prober/statemachine/SJISSMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class SJISSMModel extends SMModel
47 {
48 ////////////////////////////////////////////////////////////////
49 // constants
50 ////////////////////////////////////////////////////////////////
51 public static final int SJIS_CLASS_FACTOR = 6;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public SJISSMModel()
58 {
59 super(
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, sjisClassTable),
61 SJIS_CLASS_FACTOR,
62 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, sjisStateTable),
63 sjisCharLenTable,
64 Constants.CHARSET_SHIFT_JIS
65 );
66 }
67
68
69 ////////////////////////////////////////////////////////////////
70 // constants continued
71 ////////////////////////////////////////////////////////////////
72 private static int[] sjisClassTable = new int[] {
73 // PkgInt.pack4bits(0,1,1,1,1,1,1,1), // 00 - 07
74 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 00 - 07
75 PkgInt.pack4bits(1,1,1,1,1,1,0,0), // 08 - 0f
76 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 10 - 17
77 PkgInt.pack4bits(1,1,1,0,1,1,1,1), // 18 - 1f
78 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 20 - 27
79 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 28 - 2f
80 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 30 - 37
81 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 38 - 3f
82 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 40 - 47
83 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 48 - 4f
84 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 50 - 57
85 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 58 - 5f
86 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 60 - 67
87 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 68 - 6f
88 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // 70 - 77
89 PkgInt.pack4bits(2,2,2,2,2,2,2,1), // 78 - 7f
90 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // 80 - 87
91 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // 88 - 8f
92 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // 90 - 97
93 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // 98 - 9f
94 // 0xa0 is illegal in sjis encoding, but some pages does
95 // contain such byte. We need to be more error forgiven.
96 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a0 - a7
97 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // a8 - af
98 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b0 - b7
99 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // b8 - bf
100 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c0 - c7
101 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // c8 - cf
102 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d0 - d7
103 PkgInt.pack4bits(2,2,2,2,2,2,2,2), // d8 - df
104 PkgInt.pack4bits(3,3,3,3,3,3,3,3), // e0 - e7
105 PkgInt.pack4bits(3,3,3,3,3,4,4,4), // e8 - ef
106 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // f0 - f7
107 PkgInt.pack4bits(4,4,4,4,4,0,0,0) // f8 - ff
108 };
109
110 private static int[] sjisStateTable = new int[] {
111 PkgInt.pack4bits(ERROR,START,START, 3,ERROR,ERROR,ERROR,ERROR),//00-07
112 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ITSME,ITSME,ITSME),//08-0f
113 PkgInt.pack4bits(ITSME,ITSME,ERROR,ERROR,START,START,START,START) //10-17
114 };
115
116 private static int[] sjisCharLenTable = new int[] {
117 0, 1, 1, 2, 0, 0
118 };
119 }
+0
-97
src/org/mozilla/universalchardet/prober/statemachine/SMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 public abstract class SMModel
40 {
41 ////////////////////////////////////////////////////////////////
42 // constants
43 ////////////////////////////////////////////////////////////////
44 public static final int START = 0;
45 public static final int ERROR = 1;
46 public static final int ITSME = 2;
47
48
49 ////////////////////////////////////////////////////////////////
50 // fields
51 ////////////////////////////////////////////////////////////////
52 protected PkgInt classTable;
53 protected int classFactor;
54 protected PkgInt stateTable;
55 protected int[] charLenTable;
56 protected String name;
57
58
59 ////////////////////////////////////////////////////////////////
60 // methods
61 ////////////////////////////////////////////////////////////////
62 public SMModel(
63 PkgInt classTable,
64 int classFactor,
65 PkgInt stateTable,
66 int[] charLenTable,
67 String name)
68 {
69 this.classTable = classTable;
70 this.classFactor = classFactor;
71 this.stateTable = stateTable;
72 this.charLenTable = charLenTable;
73 this.name = name;
74 }
75
76 public int getClass(byte b)
77 {
78 int c = b & 0xFF;
79 return this.classTable.unpack(c);
80 }
81
82 public int getNextState(int cls, int currentState)
83 {
84 return this.stateTable.unpack(currentState * this.classFactor + cls);
85 }
86
87 public int getCharLen(int cls)
88 {
89 return this.charLenTable[cls];
90 }
91
92 public String getName()
93 {
94 return this.name;
95 }
96 }
+0
-121
src/org/mozilla/universalchardet/prober/statemachine/UCS2BESMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class UCS2BESMModel extends SMModel
47 {
48 ////////////////////////////////////////////////////////////////
49 // constants
50 ////////////////////////////////////////////////////////////////
51 public static final int UCS2BE_CLASS_FACTOR = 6;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public UCS2BESMModel()
58 {
59 super(
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, ucs2beClassTable),
61 UCS2BE_CLASS_FACTOR,
62 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, ucs2beStateTable),
63 ucs2beCharLenTable,
64 Constants.CHARSET_UTF_16BE
65 );
66 }
67
68
69 ////////////////////////////////////////////////////////////////
70 // constants continued
71 ////////////////////////////////////////////////////////////////
72 private static int[] ucs2beClassTable = new int[] {
73 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 00 - 07
74 PkgInt.pack4bits(0,0,1,0,0,2,0,0), // 08 - 0f
75 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 10 - 17
76 PkgInt.pack4bits(0,0,0,3,0,0,0,0), // 18 - 1f
77 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 20 - 27
78 PkgInt.pack4bits(0,3,3,3,3,3,0,0), // 28 - 2f
79 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 30 - 37
80 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 38 - 3f
81 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 40 - 47
82 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 48 - 4f
83 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 50 - 57
84 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 58 - 5f
85 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 60 - 67
86 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 68 - 6f
87 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 70 - 77
88 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 78 - 7f
89 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 80 - 87
90 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 88 - 8f
91 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 90 - 97
92 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // 98 - 9f
93 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // a0 - a7
94 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // a8 - af
95 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // b0 - b7
96 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // b8 - bf
97 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // c0 - c7
98 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // c8 - cf
99 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // d0 - d7
100 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // d8 - df
101 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // e0 - e7
102 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // e8 - ef
103 PkgInt.pack4bits(0,0,0,0,0,0,0,0), // f0 - f7
104 PkgInt.pack4bits(0,0,0,0,0,0,4,5) // f8 - ff
105 };
106
107 private static int[] ucs2beStateTable = new int[] {
108 PkgInt.pack4bits( 5, 7, 7,ERROR, 4, 3,ERROR,ERROR),//00-07
109 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ITSME,ITSME,ITSME,ITSME),//08-0f
110 PkgInt.pack4bits(ITSME,ITSME, 6, 6, 6, 6,ERROR,ERROR),//10-17
111 PkgInt.pack4bits( 6, 6, 6, 6, 6,ITSME, 6, 6),//18-1f
112 PkgInt.pack4bits( 6, 6, 6, 6, 5, 7, 7,ERROR),//20-27
113 PkgInt.pack4bits( 5, 8, 6, 6,ERROR, 6, 6, 6),//28-2f
114 PkgInt.pack4bits( 6, 6, 6, 6,ERROR,ERROR,START,START) //30-37
115 };
116
117 private static int[] ucs2beCharLenTable = new int[] {
118 2, 2, 2, 0, 2, 2
119 };
120 }
+0
-142
src/org/mozilla/universalchardet/prober/statemachine/UTF8SMModel.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in> (Java port)
22 *
23 * Alternatively, the contents of this file may be used under the terms of
24 * either the GNU General Public License Version 2 or later (the "GPL"), or
25 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
26 * in which case the provisions of the GPL or the LGPL are applicable instead
27 * of those above. If you wish to allow use of your version of this file only
28 * under the terms of either the GPL or the LGPL, and not to allow others to
29 * use your version of this file under the terms of the MPL, indicate your
30 * decision by deleting the provisions above and replace them with the notice
31 * and other provisions required by the GPL or the LGPL. If you do not delete
32 * the provisions above, a recipient may use your version of this file under
33 * the terms of any one of the MPL, the GPL or the LGPL.
34 *
35 * ***** END LICENSE BLOCK ***** */
36
37 package org.mozilla.universalchardet.prober.statemachine;
38
39 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.INDEX_SHIFT_4BITS;
40 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.SHIFT_MASK_4BITS;
41 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.BIT_SHIFT_4BITS;
42 import static org.mozilla.universalchardet.prober.statemachine.PkgInt.UNIT_MASK_4BITS;
43 import org.mozilla.universalchardet.Constants;
44
45
46 public class UTF8SMModel extends SMModel
47 {
48 ////////////////////////////////////////////////////////////////
49 // constants
50 ////////////////////////////////////////////////////////////////
51 public static final int UTF8_CLASS_FACTOR = 16;
52
53
54 ////////////////////////////////////////////////////////////////
55 // methods
56 ////////////////////////////////////////////////////////////////
57 public UTF8SMModel()
58 {
59 super(
60 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, utf8ClassTable),
61 UTF8_CLASS_FACTOR,
62 new PkgInt(INDEX_SHIFT_4BITS, SHIFT_MASK_4BITS, BIT_SHIFT_4BITS, UNIT_MASK_4BITS, utf8StateTable),
63 utf8CharLenTable,
64 Constants.CHARSET_UTF_8
65 );
66 }
67
68
69 ////////////////////////////////////////////////////////////////
70 // constants continued
71 ////////////////////////////////////////////////////////////////
72 private static int[] utf8ClassTable = new int[] {
73 // PkgInt.pack4bits(0,1,1,1,1,1,1,1), // 00 - 07
74 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 00 - 07 //allow 0x00 as a legal value
75 PkgInt.pack4bits(1,1,1,1,1,1,0,0), // 08 - 0f
76 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 10 - 17
77 PkgInt.pack4bits(1,1,1,0,1,1,1,1), // 18 - 1f
78 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 20 - 27
79 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 28 - 2f
80 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 30 - 37
81 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 38 - 3f
82 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 40 - 47
83 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 48 - 4f
84 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 50 - 57
85 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 58 - 5f
86 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 60 - 67
87 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 68 - 6f
88 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 70 - 77
89 PkgInt.pack4bits(1,1,1,1,1,1,1,1), // 78 - 7f
90 PkgInt.pack4bits(2,2,2,2,3,3,3,3), // 80 - 87
91 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 88 - 8f
92 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 90 - 97
93 PkgInt.pack4bits(4,4,4,4,4,4,4,4), // 98 - 9f
94 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // a0 - a7
95 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // a8 - af
96 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // b0 - b7
97 PkgInt.pack4bits(5,5,5,5,5,5,5,5), // b8 - bf
98 PkgInt.pack4bits(0,0,6,6,6,6,6,6), // c0 - c7
99 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // c8 - cf
100 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // d0 - d7
101 PkgInt.pack4bits(6,6,6,6,6,6,6,6), // d8 - df
102 PkgInt.pack4bits(7,8,8,8,8,8,8,8), // e0 - e7
103 PkgInt.pack4bits(8,8,8,8,8,9,8,8), // e8 - ef
104 PkgInt.pack4bits(10,11,11,11,11,11,11,11), // f0 - f7
105 PkgInt.pack4bits(12,13,13,13,14,15,0,0) // f8 - ff
106 };
107
108 private static int[] utf8StateTable = new int[] {
109 PkgInt.pack4bits(ERROR,START,ERROR,ERROR,ERROR,ERROR, 12, 10),//00-07
110 PkgInt.pack4bits( 9, 11, 8, 7, 6, 5, 4, 3),//08-0f
111 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//10-17
112 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//18-1f
113 PkgInt.pack4bits(ITSME,ITSME,ITSME,ITSME,ITSME,ITSME,ITSME,ITSME),//20-27
114 PkgInt.pack4bits(ITSME,ITSME,ITSME,ITSME,ITSME,ITSME,ITSME,ITSME),//28-2f
115 PkgInt.pack4bits(ERROR,ERROR, 5, 5, 5, 5,ERROR,ERROR),//30-37
116 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//38-3f
117 PkgInt.pack4bits(ERROR,ERROR,ERROR, 5, 5, 5,ERROR,ERROR),//40-47
118 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//48-4f
119 PkgInt.pack4bits(ERROR,ERROR, 7, 7, 7, 7,ERROR,ERROR),//50-57
120 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//58-5f
121 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR, 7, 7,ERROR,ERROR),//60-67
122 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//68-6f
123 PkgInt.pack4bits(ERROR,ERROR, 9, 9, 9, 9,ERROR,ERROR),//70-77
124 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//78-7f
125 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR, 9,ERROR,ERROR),//80-87
126 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//88-8f
127 PkgInt.pack4bits(ERROR,ERROR, 12, 12, 12, 12,ERROR,ERROR),//90-97
128 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//98-9f
129 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR, 12,ERROR,ERROR),//a0-a7
130 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//a8-af
131 PkgInt.pack4bits(ERROR,ERROR, 12, 12, 12,ERROR,ERROR,ERROR),//b0-b7
132 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR),//b8-bf
133 PkgInt.pack4bits(ERROR,ERROR,START,START,START,START,ERROR,ERROR),//c0-c7
134 PkgInt.pack4bits(ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR,ERROR) //c8-cf
135 };
136
137 private static int[] utf8CharLenTable = new int[] {
138 0, 1, 0, 0, 0, 0, 2, 3,
139 3, 3, 4, 4, 5, 5, 6, 6
140 };
141 }
+45
-0
src/test/java/org/mozilla/universalchardet/BOMTest.java less more
0 package org.mozilla.universalchardet;
1
2 import java.io.BufferedReader;
3 import java.io.File;
4 import java.io.IOException;
5
6 import org.junit.Assert;
7 import org.junit.Test;
8
9
10
11 public class BOMTest {
12
13 private static final String TEST_STRING =
14 "========================================================================";
15
16 @Test
17 public void testUTF8 () throws IOException {
18 Assert.assertEquals(TEST_STRING, getFirstLine("utf8.txt"));
19 }
20 @Test
21 public void testUTF8N () throws IOException {
22 Assert.assertEquals(TEST_STRING, getFirstLine("utf8n.txt"));
23 }
24 @Test
25 public void testUTF16LE () throws IOException {
26 Assert.assertEquals(TEST_STRING, getFirstLine("utf16le.txt"));
27 }
28
29 private String getFirstLine(String testFileName) throws IOException{
30 String fileName = "src/test/resources/" + testFileName;
31 File file = new File(fileName);
32 BufferedReader reader = null;
33 try {
34 reader = ReaderFactory.createBufferedReader(file);
35 // return first line
36 return reader.readLine();
37 }
38 finally {
39 if (reader != null) {
40 reader.close();
41 }
42 }
43 }
44 }
+79
-0
src/test/java/org/mozilla/universalchardet/BasicFileEncodingDetectionTest.java less more
0 package org.mozilla.universalchardet;
1
2 import java.io.File;
3 import java.io.IOException;
4
5 import org.junit.Assert;
6 import org.junit.Test;
7
8 public class BasicFileEncodingDetectionTest {
9
10
11 public BasicFileEncodingDetectionTest() throws IOException {
12 super();
13 }
14
15
16 @Test
17 public void testASCII() throws IOException {
18 Assert.assertEquals("US-ASCII", getFileEncoding("ascii.txt"));
19 }
20
21 @Test
22 public void testUTF8 () throws IOException {
23 Assert.assertEquals("UTF-8", getFileEncoding("utf8.txt"));
24 }
25 @Test
26 public void testUTF8N () throws IOException {
27 Assert.assertEquals("UTF-8", getFileEncoding("utf8n.txt"));
28 }
29 @Test
30 public void testUTF16LE () throws IOException {
31 Assert.assertEquals("UTF-16LE", getFileEncoding("utf16le.txt"));
32 }
33 @Test
34 public void testShifJis () throws IOException {
35 Assert.assertEquals("SHIFT_JIS", getFileEncoding("shiftjis.txt"));
36 }
37
38 @Test
39 public void testEUC () throws IOException {
40 Assert.assertEquals("EUC-JP", getFileEncoding("euc.txt"));
41 }
42 @Test
43 public void testISO2022JP () throws IOException {
44 Assert.assertEquals("ISO-2022-JP", getFileEncoding("iso2022jp.txt"));
45 }
46
47
48
49 @Test
50 public void testBIG5 () throws IOException {
51 Assert.assertEquals("BIG5", getFileEncoding("big5.txt"));
52 }
53
54 @Test
55 public void testEUCTW () throws IOException {
56 Assert.assertEquals("EUC-TW", getFileEncoding("euctw.txt"));
57 }
58 @Test
59 public void testEUCKR() throws IOException {
60 Assert.assertEquals("EUC-KR", getFileEncoding("euckr.txt"));
61 }
62 @Test
63 public void testWindows1255 () throws IOException {
64 Assert.assertEquals("WINDOWS-1255", getFileEncoding("windows1255.txt"));
65 }
66
67
68 @Test
69 public void testUTF8Emoji () throws IOException {
70 Assert.assertEquals("UTF-8", getFileEncoding("utf8n-emoji.txt"));
71 }
72
73
74 private String getFileEncoding(String testFileName) throws IOException{
75 String fileName = "src/test/resources/" + testFileName;
76 return UniversalDetector.detectCharset(new File(fileName));
77 }
78 }
+88
-0
src/test/java/org/mozilla/universalchardet/BasicStreamEncodingDetectionTest.java less more
0 package org.mozilla.universalchardet;
1
2 import java.io.BufferedInputStream;
3 import java.io.File;
4 import java.io.IOException;
5 import java.nio.file.Files;
6
7 import org.junit.Assert;
8 import org.junit.Test;
9
10 public class BasicStreamEncodingDetectionTest {
11 @Test
12 public void testUTF8 () throws IOException {
13 Assert.assertEquals("UTF-8", getFileEncoding("utf8.txt"));
14 }
15 @Test
16 public void testUTF8N () throws IOException {
17 Assert.assertEquals("UTF-8", getFileEncoding("utf8n.txt"));
18 }
19 @Test
20 public void testUTF16LE () throws IOException {
21 Assert.assertEquals("UTF-16LE", getFileEncoding("utf16le.txt"));
22 }
23 @Test
24 public void testShifJis () throws IOException {
25 Assert.assertEquals("SHIFT_JIS", getFileEncoding("shiftjis.txt"));
26 }
27
28 @Test
29 public void testEUC () throws IOException {
30 Assert.assertEquals("EUC-JP", getFileEncoding("euc.txt"));
31 }
32 @Test
33 public void testISO2022JP () throws IOException {
34 Assert.assertEquals("ISO-2022-JP", getFileEncoding("iso2022jp.txt"));
35 }
36
37 @Test
38 public void testBIG5 () throws IOException {
39 Assert.assertEquals("BIG5", getFileEncoding("big5.txt"));
40 }
41
42 @Test
43 public void testEUCTW () throws IOException {
44 Assert.assertEquals("EUC-TW", getFileEncoding("euctw.txt"));
45 }
46 @Test
47 public void testEUCKR() throws IOException {
48 Assert.assertEquals("EUC-KR", getFileEncoding("euckr.txt"));
49 }
50 @Test
51 public void testWindows1255 () throws IOException {
52 Assert.assertEquals("WINDOWS-1255", getFileEncoding("windows1255.txt"));
53 }
54
55 @Test
56 public void testUTF8Emoji () throws IOException {
57 Assert.assertEquals("UTF-8", getFileEncoding("utf8n-emoji.txt"));
58 }
59
60
61
62 private String getFileEncoding(String testFileName) throws IOException{
63 File file = new File("src/test/resources/" + testFileName);
64 EncodingDetectorInputStream edis = null;
65 EncodingDetectorOutputStream edos = null;
66 try {
67 edis = new EncodingDetectorInputStream(new BufferedInputStream(Files.newInputStream(file.toPath())));
68 edos = new EncodingDetectorOutputStream(NullOutputStream.NULL_OUTPUT_STREAM);
69 byte[] buffer = new byte[1024];
70 int read = 0;
71 while (( read = edis.read(buffer)) > 0){
72 edos.write(buffer, 0, read);
73 }
74 }
75 finally {
76 edos.close();
77 edis.close();
78 }
79 String encodingRead = edis.getDetectedCharset();
80 String encodingWrite = edos.getDetectedCharset();
81 Assert.assertNotNull(encodingRead);
82 Assert.assertNotNull(encodingWrite);
83 Assert.assertTrue(encodingRead.equals(encodingWrite));
84 return encodingRead;
85
86 }
87 }
+46
-0
src/test/java/org/mozilla/universalchardet/Bug20LatinDetectedAsMaccyrillicTest.java less more
0 package org.mozilla.universalchardet;
1
2 import java.io.File;
3 import java.io.IOException;
4 import java.nio.charset.Charset;
5
6 import org.junit.Assert;
7 import org.junit.Ignore;
8 import org.junit.Test;
9
10 public class Bug20LatinDetectedAsMaccyrillicTest {
11
12 private static final String TEST_STRING = "ÄÜÖßäöü,Name1ÄÜÖßäöü,Name2ÄÜÖßäöü,Name3ÄÜÖßäöü,StreetÄÜÖßäöü,MÄÜÖßäöü,DE,80080,München,ContactÄÜÖßäöü,+49(0)ÄÜÖßäöü,ÄÜÖßäöü@gls-itservices.com,CommentÄÜÖßäöü,+49,(0)98,765,432,BlÄÜÖßäöü";
13
14
15
16 @Test
17 @Ignore("Bug not fixed yet")
18 public void testFile () throws IOException {
19 File testFile = new File("src/test/resources/bug20-example-latin.txt");
20 String originalEncoding = UniversalDetector.detectCharset(testFile);
21 Assert.assertEquals("WINDOWS-1252", originalEncoding);
22 }
23
24 @Test
25 @Ignore("Bug not fixed yet")
26 public void testLatin() throws IOException {
27
28 UniversalDetector detector = new UniversalDetector();
29 detector.handleData(TEST_STRING.getBytes(Charset.forName("WINDOWS-1252")));
30 detector.dataEnd();
31 Assert.assertEquals("WINDOWS-1252", detector.getDetectedCharset());
32
33 }
34
35
36 @Test
37 public void testUTF8() {
38 UniversalDetector detector = new UniversalDetector();
39 detector.handleData(TEST_STRING.getBytes(Charset.forName("UTF-8")));
40 detector.dataEnd();
41 Assert.assertEquals("UTF-8", detector.getDetectedCharset());
42
43 }
44
45 }
+34
-0
src/test/java/org/mozilla/universalchardet/Bug33USASCIIToGenerous.java less more
0 package org.mozilla.universalchardet;
1
2 import java.io.ByteArrayInputStream;
3 import java.io.IOException;
4
5 import org.junit.Assert;
6 import org.junit.Ignore;
7 import org.junit.Test;
8
9 public class Bug33USASCIIToGenerous {
10
11 public Bug33USASCIIToGenerous() {
12 super();
13 }
14
15
16
17 @Test
18 @Ignore("Not sure")
19 public void testUTF16 () throws IOException {
20 Assert.assertEquals("UTF-16BE", detect("ab".getBytes("UTF-16BE")));
21 Assert.assertEquals("UTF-16LE", detect("ab".getBytes("UTF-16LE")));
22 }
23 @Test
24 public void testZipHeader() throws IOException {
25 byte[] zipHeader = new byte[]{0x50, 0x4b, 0x03, 0x04, 0x14, 0x00, 0x02, 0x00};
26 Assert.assertNull(detect(zipHeader));
27 }
28
29 private String detect(byte[] data) throws IOException {
30 return UniversalDetector.detectCharset(new ByteArrayInputStream(data));
31 }
32
33 }
+70
-0
src/test/java/org/mozilla/universalchardet/Bug8VariousFailedCharsets.java less more
0 package org.mozilla.universalchardet;
1
2 import org.junit.Assert;
3 import org.junit.Ignore;
4 import org.junit.Test;
5
6
7 /*
8 * This charset detection fails because too little data to analize
9 */
10 public class Bug8VariousFailedCharsets {
11
12
13
14 @Test
15 @Ignore
16 public void test1() throws Exception {
17 /*
18 =?KR?B?wPzDvCCwy7v2?=
19 전체 검색
20 */
21
22 byte[] data = decodeBase64("wPzDvCCwy7v2");
23 String charset = detect(data);
24 Assert.assertNotNull(charset);
25 Assert.assertEquals("전체 검색", new String(data, charset));
26 }
27
28 @Test
29 public void test2() throws Exception {
30 /*
31 =?wws-874?B?W1Nhbm9va10gt7TKzbq70a3L0qHSw83o0rnA0snS5LfCzdWhpMPR6acg8fLz?=
32 [Sanook] ทดสอบปัญหาการอ่านภาษาไทยอีกครั้ง ๑๒๓
33 */
34
35 byte[] data = decodeBase64("W1Nhbm9va10gt7TKzbq70a3L0qHSw83o0rnA0snS5LfCzdWhpMPR6acg8fLz");
36 String charset = detect(data);
37 Assert.assertNotNull(charset);
38 Assert.assertEquals("[Sanook] ทดสอบปัญหาการอ่านภาษาไทยอีกครั้ง ๑๒๓", new String(data, charset));
39 }
40
41 @Test
42 @Ignore
43 public void test3() throws Exception {
44 /*
45 =?gb2312?B?zfjS19PKz+TX1Lavu9i4tDo=?= Re: �˲ű�
46 网易邮箱自动回复: Re: 人才表
47 */
48 byte[] data = decodeBase64("zfjS19PKz+TX1Lavu9i4tDo=");
49 String charset = detect(data);
50 Assert.assertNotNull(charset);
51 Assert.assertEquals("网易邮箱自动回复:", new String(data, charset));
52
53
54
55 }
56 private String detect(byte[] data) {
57 UniversalDetector detector = new UniversalDetector();
58 detector.handleData(data);
59 detector.dataEnd();
60 String detected = detector.getDetectedCharset();
61 detector.reset();
62
63 return detected;
64 }
65 private byte[] decodeBase64(String string) {
66 return org.apache.commons.codec.binary.Base64.decodeBase64(string);
67 }
68
69 }
+54
-0
src/test/java/org/mozilla/universalchardet/GB18030SMFalsePositiveTest.java less more
0 package org.mozilla.universalchardet;
1
2 import static org.junit.Assert.assertEquals;
3
4 import java.io.UnsupportedEncodingException;
5
6 import org.junit.Assert;
7 import org.junit.Ignore;
8 import org.junit.Test;
9
10 /**
11 * @author ian
12 * @since Jul 13, 2011
13 *
14 */
15 public class GB18030SMFalsePositiveTest {
16
17 public GB18030SMFalsePositiveTest() {
18 super();
19 }
20
21 @Test
22 @Ignore
23 public void testFalsePositiveBug11() throws UnsupportedEncodingException {
24 String testString = "[°4°0°T°C°C°0°C°T";
25 byte[] testBuf = new byte[]{91, -80, 52, -80, 48, -80, 84, -80, 67, -80, 67, -80, 48, -80, 67, -80, 84};
26 byte[] buf = testString.getBytes("WINDOWS-1252");
27 Assert.assertArrayEquals(testBuf, buf);
28
29 UniversalDetector detector = new UniversalDetector();
30 detector.handleData(buf, 0, buf.length);
31 detector.dataEnd();
32
33 String encoding = detector.getDetectedCharset();
34 detector.reset();
35 assertEquals("WINDOWS-1252", encoding);
36 }
37
38 @Test
39 @Ignore
40 public void testFalsePositiveBug9() throws UnsupportedEncodingException {
41 String testString = "Wykamol,£588.95,0.18,0.12,testingSpecialised Products for DIY and Professionals£12";
42 byte[] buf = testString.getBytes("WINDOWS-1252");
43
44 UniversalDetector detector = new UniversalDetector();
45 detector.handleData(buf);
46 detector.dataEnd();
47
48 String encoding = detector.getDetectedCharset();
49 detector.reset();
50
51 assertEquals("WINDOWS-1252", encoding);
52 }
53 }⏎
+29
-0
src/test/java/org/mozilla/universalchardet/GB18030SamplesTest.java less more
0 package org.mozilla.universalchardet;
1
2 import java.io.File;
3 import java.io.IOException;
4
5 import org.junit.Assert;
6 import org.junit.Test;
7
8 public class GB18030SamplesTest {
9
10 public GB18030SamplesTest() {
11 super();
12 }
13
14 @Test
15 public void testGB18030Sample() throws IOException {
16 Assert.assertEquals("GB18030", getFileEncoding("gb2312-sample.txt"));
17 }
18
19 @Test
20 public void testGBKSample() throws IOException {
21 Assert.assertEquals("GB18030", getFileEncoding("gbk-sample.txt"));
22 }
23
24 private String getFileEncoding(String testFileName) throws IOException{
25 String fileName = "src/test/resources/" + testFileName;
26 return UniversalDetector.detectCharset(new File(fileName));
27 }
28 }
+49
-0
src/test/java/org/mozilla/universalchardet/NullOutputStream.java less more
0 package org.mozilla.universalchardet;
1
2 import java.io.IOException;
3 import java.io.OutputStream;
4
5 /**
6 * This OutputStream writes all data to the famous <b>/dev/null</b>.
7 * <p>
8 * This output stream has no destination (file/socket etc.) and all
9 * bytes written to it are ignored and lost.
10 *
11 * @author Jeremias Maerki
12 * @version $Id: NullOutputStream.java 610010 2008-01-08 14:50:59Z niallp $
13 */
14 public class NullOutputStream extends OutputStream {
15
16 /**
17 * A singleton.
18 */
19 public static final NullOutputStream NULL_OUTPUT_STREAM = new NullOutputStream();
20
21 /**
22 * Does nothing - output to <code>/dev/null</code>.
23 * @param b The bytes to write
24 * @param off The start offset
25 * @param len The number of bytes to write
26 */
27 public void write(byte[] b, int off, int len) {
28 //to /dev/null
29 }
30
31 /**
32 * Does nothing - output to <code>/dev/null</code>.
33 * @param b The byte to write
34 */
35 public void write(int b) {
36 //to /dev/null
37 }
38
39 /**
40 * Does nothing - output to <code>/dev/null</code>.
41 * @param b The bytes to write
42 * @throws IOException never
43 */
44 public void write(byte[] b) throws IOException {
45 //to /dev/null
46 }
47
48 }
+63
-0
src/test/java/org/mozilla/universalchardet/ReaderTest.java less more
0 package org.mozilla.universalchardet;
1
2 import java.io.BufferedReader;
3 import java.io.File;
4 import java.io.IOException;
5
6 import org.junit.Assert;
7 import org.junit.Test;
8
9 public class ReaderTest {
10
11 private static final String TEST_STRING =
12 " コンソール アプリケーション : universalchardet プロジェクトの概要";
13
14
15
16 @Test
17 public void testUTF8 () throws IOException {
18 Assert.assertEquals(TEST_STRING, getSecondLine("utf8.txt"));
19 }
20 @Test
21 public void testUTF8N () throws IOException {
22 Assert.assertEquals(TEST_STRING, getSecondLine("utf8n.txt"));
23 }
24 @Test
25 public void testUTF16LE () throws IOException {
26 Assert.assertEquals(TEST_STRING, getSecondLine("utf16le.txt"));
27 }
28 @Test
29 public void testShifJis () throws IOException {
30 Assert.assertEquals(TEST_STRING, getSecondLine("shiftjis.txt"));
31 }
32
33 @Test
34 public void testEUC () throws IOException {
35 Assert.assertEquals(TEST_STRING, getSecondLine("euc.txt"));
36 }
37 @Test
38 public void testISO2022JP () throws IOException {
39 Assert.assertEquals(TEST_STRING, getSecondLine("iso2022jp.txt"));
40 }
41
42
43
44 private String getSecondLine(String testFileName) throws IOException{
45 String fileName = "src/test/resources/" + testFileName;
46 File file = new File(fileName);
47 BufferedReader reader = null;
48 try {
49 reader = ReaderFactory.createBufferedReader(file);
50 // Skip first line
51 reader.readLine();
52 // return second line
53 return reader.readLine();
54 }
55 finally {
56 if (reader != null) {
57 reader.close();
58 }
59 }
60 }
61
62 }
+70
-0
src/test/java/org/mozilla/universalchardet/ShortStringTests.java less more
0 package org.mozilla.universalchardet;
1
2 import java.io.UnsupportedEncodingException;
3 import java.nio.charset.Charset;
4 import java.nio.charset.StandardCharsets;
5
6 import org.junit.Assert;
7 import org.junit.Ignore;
8 import org.junit.Test;
9
10 public class ShortStringTests {
11
12 public ShortStringTests() {
13 super();
14 }
15
16 // Tets case for https://github.com/albfernandez/juniversalchardet/issues/22
17 @Test
18 @Ignore("data too short")
19 public void testDecodeBytes() {
20
21 final String string = "aeaCàêäÇ";
22 Charset s;
23 byte[] bytes;
24
25 bytes = string.getBytes(StandardCharsets.UTF_8);
26 s = this.guessCharset(bytes);
27 Assert.assertEquals(string, new String(string.getBytes(s), s)); // SUCCESS
28
29 bytes = string.getBytes(StandardCharsets.ISO_8859_1);
30 s = this.guessCharset(bytes); // detected charset = TIS-620, Thai charset ???!!!
31 Assert.assertEquals(string, new String(string.getBytes(s), s)); // FAILS of course !
32 }
33
34 // Tets case for https://github.com/albfernandez/juniversalchardet/issues/22
35 // With less accute characters, it's improved detection
36 @Test
37 public void testDecodeBytesBetterStats() {
38
39 final String string = "Château";
40 Charset s;
41 byte[] bytes;
42
43 bytes = string.getBytes(StandardCharsets.UTF_8);
44 s = this.guessCharset(bytes);
45 Assert.assertEquals(string, new String(string.getBytes(s), s)); // SUCCESS
46
47 bytes = string.getBytes(StandardCharsets.ISO_8859_1);
48 s = this.guessCharset(bytes);
49 Assert.assertEquals(string, new String(string.getBytes(s), s)); // SUCCESS
50 }
51
52
53 @Test
54 public void testShortString() throws UnsupportedEncodingException {
55 Assert.assertEquals("US-ASCII", guessCharsetName("abcd".getBytes()));
56 // Assert.assertNull(guessCharsetName("Ábcd".getBytes("ISO-8859-15")));
57 }
58
59 private Charset guessCharset(final byte[] bytes) {
60 return Charset.forName(guessCharsetName(bytes));
61 }
62
63 private String guessCharsetName(final byte[] bytes) {
64 final UniversalDetector detector = new UniversalDetector();
65 detector.handleData(bytes, 0, bytes.length);
66 detector.dataEnd();
67 return detector.getDetectedCharset();
68 }
69 }
+22
-0
src/test/java/org/mozilla/universalchardet/TIS620BasicTest.java less more
0 package org.mozilla.universalchardet;
1
2 import java.io.File;
3 import java.io.IOException;
4
5 import org.junit.Assert;
6 import org.junit.Test;
7
8 public class TIS620BasicTest {
9
10 @Test
11 public void testTIS620() throws IOException {
12 Assert.assertEquals("TIS620", getFileEncoding("tis620.txt"));
13 }
14
15
16
17 private String getFileEncoding(String testFileName) throws IOException{
18 String fileName = "src/test/resources/" + testFileName;
19 return UniversalDetector.detectCharset(new File(fileName));
20 }
21 }
+28
-0
src/test/java/org/mozilla/universalchardet/example/TestCreateReaderFromFile.java less more
0 package org.mozilla.universalchardet.example;
1
2 import org.mozilla.universalchardet.ReaderFactory;
3
4 public class TestCreateReaderFromFile {
5
6 public static void main (String[] args) throws java.io.IOException {
7 if (args.length != 1) {
8 System.err.println("Usage: java TestCreateReaderFromFile FILENAME");
9 System.exit(1);
10 }
11
12 java.io.Reader reader = null;
13 try {
14 java.io.File file = new java.io.File(args[0]);
15 reader = ReaderFactory.createBufferedReader(file);
16
17 // Do whatever you want with the reader
18 }
19 finally {
20 if (reader != null) {
21 reader.close();
22 }
23 }
24
25 }
26
27 }
+80
-0
src/test/java/org/mozilla/universalchardet/example/TestDetector.java less more
0 /* ***** BEGIN LICENSE BLOCK *****
1 * Version: MPL 1.1/GPL 2.0/LGPL 2.1
2 *
3 * The contents of this file are subject to the Mozilla Public License Version
4 * 1.1 (the "License"); you may not use this file except in compliance with
5 * the License. You may obtain a copy of the License at
6 * http://www.mozilla.org/MPL/
7 *
8 * Software distributed under the License is distributed on an "AS IS" basis,
9 * WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
10 * for the specific language governing rights and limitations under the
11 * License.
12 *
13 * The Original Code is mozilla.org code.
14 *
15 * The Initial Developer of the Original Code is
16 * Netscape Communications Corporation.
17 * Portions created by the Initial Developer are Copyright (C) 1998
18 * the Initial Developer. All Rights Reserved.
19 *
20 * Contributor(s):
21 * Kohei TAKETA <k-tak@void.in>
22 * lnezda
23 *
24 * Alternatively, the contents of this file may be used under the terms of
25 * either the GNU General Public License Version 2 or later (the "GPL"), or
26 * the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
27 * in which case the provisions of the GPL or the LGPL are applicable instead
28 * of those above. If you wish to allow use of your version of this file only
29 * under the terms of either the GPL or the LGPL, and not to allow others to
30 * use your version of this file under the terms of the MPL, indicate your
31 * decision by deleting the provisions above and replace them with the notice
32 * and other provisions required by the GPL or the LGPL. If you do not delete
33 * the provisions above, a recipient may use your version of this file under
34 * the terms of any one of the MPL, the GPL or the LGPL.
35 *
36 * ***** END LICENSE BLOCK ***** */
37 package org.mozilla.universalchardet.example;
38
39 import java.io.InputStream;
40 import java.nio.file.Files;
41 import java.nio.file.Paths;
42
43 import org.mozilla.universalchardet.UniversalDetector;
44
45 public class TestDetector {
46 public static void main(String[] args) throws java.io.IOException {
47 if (args.length != 1) {
48 System.err.println("Usage: java TestDetector FILENAME");
49 System.exit(1);
50 }
51
52 byte[] buf = new byte[4096];
53 String fileName = args[0];
54 try (InputStream fis = Files.newInputStream(Paths.get(fileName))) {
55
56 // (1)
57 UniversalDetector detector = new UniversalDetector();
58
59 // (2)
60 int nread;
61 while ((nread = fis.read(buf)) > 0 && !detector.isDone()) {
62 detector.handleData(buf, 0, nread);
63 }
64 // (3)
65 detector.dataEnd();
66
67 // (4)
68 String encoding = detector.getDetectedCharset();
69 if (encoding != null) {
70 System.out.println("Detected encoding = " + encoding);
71 } else {
72 System.out.println("No encoding detected.");
73 }
74
75 // (5)
76 detector.reset();
77 }
78 }
79 }
+20
-0
src/test/java/org/mozilla/universalchardet/example/TestDetectorFile.java less more
0 package org.mozilla.universalchardet.example;
1
2 import org.mozilla.universalchardet.UniversalDetector;
3
4 public class TestDetectorFile {
5
6 public static void main (String[] args) throws java.io.IOException {
7 if (args.length != 1) {
8 System.err.println("Usage: java TestDetectorFile FILENAME");
9 System.exit(1);
10 }
11 java.io.File file = new java.io.File(args[0]);
12 String encoding = UniversalDetector.detectCharset(file);
13 if (encoding != null) {
14 System.out.println("Detected encoding = " + encoding);
15 } else {
16 System.out.println("No encoding detected.");
17 }
18 }
19 }
+31
-0
src/test/java/org/mozilla/universalchardet/prober/statemachine/PkgIntTest.java less more
0 package org.mozilla.universalchardet.prober.statemachine;
1
2 import static org.junit.Assert.*;
3
4 import org.junit.Test;
5
6 public class PkgIntTest
7 {
8 @Test
9 public void testUnpack()
10 {
11 int[] data = new int[] {
12 PkgInt.pack4bits(0, 1, 2, 3, 4, 5, 6, 7),
13 PkgInt.pack4bits(8, 9, 10, 11, 12, 13, 14, 15)
14 };
15
16 PkgInt pkg = new PkgInt(
17 PkgInt.INDEX_SHIFT_4BITS,
18 PkgInt.SHIFT_MASK_4BITS,
19 PkgInt.BIT_SHIFT_4BITS,
20 PkgInt.UNIT_MASK_4BITS,
21 data);
22
23 for (int i=0; i<16; ++i) {
24 int n = pkg.unpack(i);
25 if (n != i) {
26 fail("Invalid packed value.");
27 }
28 }
29 }
30 }
+11
-0
src/test/resources/ascii.txt less more
0
1
2 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean massa ipsum, accumsan at tincidunt at, viverra non nisi. Praesent interdum justo at quam vehicula, vel semper ligula porta. Nam elementum est et velit gravida dignissim. Praesent bibendum bibendum leo id interdum. Vestibulum vel mollis risus, eget consequat odio. Fusce magna magna, rhoncus quis tincidunt eu, laoreet quis lacus. Nullam at ligula id lacus cursus convallis. Mauris ut vulputate mi, quis sodales quam. In diam diam, lacinia ac cursus eget, bibendum sodales risus. Sed est nibh, vestibulum ac euismod in, dictum ullamcorper odio.
3
4 Etiam at tristique est. Mauris quis erat at risus dignissim euismod. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Maecenas iaculis malesuada pharetra. Quisque semper elit vel ex pellentesque fermentum vitae accumsan tellus. Maecenas ut blandit eros. Vestibulum orci lectus, volutpat in sapien vel, lobortis blandit sapien. Etiam tristique est et eros placerat iaculis.
5
6 Phasellus ac mollis est. Suspendisse at nulla ullamcorper, tincidunt nisi quis, aliquam augue. Donec id neque purus. Duis risus neque, pharetra et laoreet bibendum, dictum ut tortor. In malesuada viverra condimentum. Donec cursus lorem risus, a consequat tellus hendrerit non. Pellentesque a feugiat mauris. Ut vel dui sed tortor pharetra tristique ac feugiat ligula. Integer viverra consectetur augue, nec blandit dolor pellentesque tincidunt. Duis ut augue sed dolor efficitur vehicula a iaculis turpis. Suspendisse ultrices, nunc in posuere sagittis, erat elit feugiat lacus, id hendrerit elit neque ut velit. Mauris eu varius massa. Duis gravida, mauris a consequat blandit, neque diam interdum justo, non tincidunt justo elit id mi. Curabitur hendrerit nisi purus, id luctus mi maximus nec.
7
8 Pellentesque in volutpat diam, sagittis varius quam. In metus mauris, aliquam gravida feugiat eu, condimentum sed augue. Cras cursus, sem ut venenatis lacinia, quam ipsum pharetra risus, eu scelerisque odio orci non turpis. In commodo, felis accumsan pellentesque tempor, ligula nisl ultrices quam, eu facilisis tellus erat sed dolor. Suspendisse et nibh id nisi aliquet tempus non sed nisi. Donec cursus purus quam, nec dictum felis tempor ac. Nunc eu est orci. Sed fringilla diam id tortor pulvinar, vel faucibus turpis malesuada. Morbi pharetra sit amet dolor eget faucibus. Nam in tristique sem. Ut et eros ac lorem bibendum mattis. Aliquam erat volutpat. Nunc congue mattis neque, eu maximus nisi pellentesque at.
9
10 Quisque laoreet faucibus justo a auctor. Nunc condimentum erat tempor, placerat nulla vel, convallis urna. Praesent rhoncus nulla orci, vitae laoreet eros elementum quis. Quisque sed pellentesque tortor, id aliquet libero. Donec ultrices rhoncus felis nec congue. Donec eu ullamcorper massa. Aliquam at auctor elit. ⏎
+10
-0
src/test/resources/big5.txt less more
0 ¤j¤­½X
1 維基百科，自由的百科全書
2
3 Big5，又稱為大五碼或五大碼，是使用繁體中文（正體中文）社群中最常用的電腦漢字字符集標準，共收錄13,060個漢字[1]。
4
5 中文碼分為內碼及交換碼兩類，Big5屬中文內碼，知名的中文交換碼有CCCII、CNS11643。
6
7 Big5雖普及於台灣、香港與澳門等繁體中文通行區，但長期以來並非當地的國家標準，而只是業界標準。倚天中文系統、Windows繁體中文版等主要系統的字符集都是以Big5為基準，但廠商又各自增加不同的造字與造字區，衍生成多種不同版本。
8
9 2003年，Big5被收錄到CNS11643中文標準交換碼的附錄當中，取得了較正式的地位。這個最新版本被稱為Big5-2003。
+1
-0
src/test/resources/bug20-example-latin.txt less more
0 umlouds,Name1ÄÜÖßäöü,Name2ÄÜÖßäöü,Name3ÄÜÖßäöü,StreetÄÜÖßäöü,MÄÜÖßäöü,DE,80080,München,ContactÄÜÖßäöü,+49(0)ÄÜÖßäöü,ÄÜÖßäöü@gls-itservices.com,CommentÄÜÖßäöü,+49,(0)98,765,432,BlÄÜÖßäöü⏎
+17
-0
src/test/resources/bug8-various-failed-charsets.dat less more
0 =?KR?B?wPzDvCCwy7v2?=
1 전체 검색
2
3 =?wws-874?B?W1Nhbm9va10gt7TKzbq70a3L0qHSw83o0rnA0snS5LfCzdWhpMPR6acg8fLz?=
4 [Sanook] ทดสอบปัญหาการอ่านภาษาไทยอีกครั้ง ๑๒๓
5
6 =?wws-874?B?W1Nhbm9va10gt7TKzbq70a3L0qHSw83o0rnA0snS5LfCzdWhpMPR6acg8fLz?=
7 [Sanook] ทดสอบปัญหาการอ่านภาษาไทยอีกครั้ง ๑๒๓
8
9 =?gb2312?B?zfjS19PKz+TX1Lavu9i4tDo=?= Re: »À≤≈±Ì
10 网易邮箱自动回复: Re: 人才表
11
12  dracioppa1216@rogers.com
13  dracioppa1216@rogers.com
14
15
16 German / Deutsch Süd Grüß Gott French / Français
+39
-0
src/test/resources/euc.txt less more
0 ========================================================================
1 コンソール アプリケーション : universalchardet プロジェクトの概要
2 ========================================================================
3
4 この universalchardet アプリケーションは、AppWizard によって作成されました。
5
6 このファイルには、universalchardet アプリケーションを構成する各ファイルの
7 内容の概略が記述されています。
8
9
10 universalchardet.vcproj
11 これは、アプリケーション ウィザードで生成される VC++ プロジェクトのメインの
12 プロジェクト ファイルです。
13 ファイルを生成した Visual C++ のバージョン情報と、アプリケーション
14 ウィザードで選択したプラットフォーム、構成、およびプロジェクトの機能に関する
15 情報が記述されています。
16
17 universalchardet.cpp
18 これは、メインのアプリケーション ソース ファイルです。
19
20 /////////////////////////////////////////////////////////////////////////////
21 その他の標準ファイル :
22
23 StdAfx.h, StdAfx.cpp
24 これらのファイルは、コンパイル済みヘッダー (PCH) ファイル
25 universalchardet.pch とプリコンパイル済み型ファイル StdAfx.obj を
26 ビルドするために使用します。
27
28 /////////////////////////////////////////////////////////////////////////////
29 その他のメモ :
30
31 AppWizard では "TODO:" コメントを使用して、ユーザーが追加またはカスタマイズする
32 ソース部分を示します。
33
34 /////////////////////////////////////////////////////////////////////////////
35
36
37
38
+12
-0
src/test/resources/euckr.txt less more
0 EUC
1 위키백과, 우리 모두의 백과사전.
2
3 EUC(Extended Unix Code, 확장 유닉스 코드)란 한국어, 중국어, 일본어 문자 전산화에 주로 사용되는 8비트 문자 인코딩 방식이다.
4
5 EUC의 구조는 ISO 2022 표준에 기반하고 있다.
6
7 대한민국의 인터넷 환경에서 광범위하게 쓰이고 있어 친숙한 EUC-KR은 이 인코딩 방식을 사용하여 한글 등 한국어에서 사용되는 문자를 표현한 것이다. 사용빈도는 다르지만 중화인민공화국에서는 EUC-CN, 중화민국(대만)에서는 EUC-TW, 일본에서는 EUC-JP 등의 인코딩 방식이 존재한다.
8 원본 주소 "https://ko.wikipedia.org/w/index.php?title=EUC&oldid=10340616"
9 ║лищ:
10
11 문자 인코딩
+10
-0
src/test/resources/euctw.txt less more
0 ƒЋƒмо£
1 êÄÙ×ÈËÒï¡¢ÈÜÆñÎûÈËÒïÇÀÕó
2
3 Big5¡¢Ä´ê¢Ò³ÄËÄìî£ÍÐÄìÄË¢ÑÒËðÆîôíüÕÄãÅÆ¡¾ÆßüÕÄãÅÆ¡¿Ï£åúÄãÞ²Ú¦ÆîÎûçÙæªéÇÇóÇóÜÊâõíºäí¡¢ÇÁÈ­òç13,060Ô¶éÇÇó[1]¡¤
4
5 ÄãÅÆî£Å¡Ò³Äùî£Å¯Ç¨ßÐî£Ì¥ùË¡¢Big5úèÄãÅÆÄù¢Ï¡ÇØÎûÄãÅÆǨßÐî£È´CCCII¡£CNS11643¡¤
6
7 Big5ö£ßÛůÍõƺý¤¡£Óüà«êØñ®ÏÜá«ôíüÕÄãÅÆÝ×ÈçÙ´¡¢ÈþÏÛßæÅèËôËäÏæåµÇâÎûÙÏÕ¥íºäí¡¢ÈÔÆ·ÑÒäÆÒÌíºäí¡¤Ô­Å²ÄãÅÆË·ÜÓ¡£WindowsôíüÕÄãÅÆÎêá«ÅäÓÑË·ÜÓÎûÇóÜÊâõÝçÑÒÅèBig5Ò³Ù×äí¡¢ÈþìÞٸĴÇÖÈÜìÁÅûÄâÇÑÎûÝßÇóêØÝßÇóÙ´¡¢ÓÏÆíÈ©Çéê¡ÄâÇÑÎêÆÛ¡¤
8
9 2003È¡¡¢Big5Ý·È­òç̯CNS11643ÄãÅÆíºäíǨßÐî£ÎûÏáòçåµÄ㡢̽ڵħç¤ÆßÈ¢ÎûÇâÈí¡¤ÝÕÔ¶Þ²ä»ÎêÆÛÝ·ê¢Ò³Big5-2003¡¤
+17
-0
src/test/resources/gb18030.txt less more
0 GB 18030
1 维基百科，自由的百科全书
2
3 GB 18030，全称：国家标准GB 18030-2005《信息技术　中文编码字符集》，是中华人民共和国现时最新的内码字集，是GB 18030-2000《信息技术　信息交换用汉字编码字符集　基本集的扩充》的修订版。与GB 2312-1980完全兼容，与GBK基本兼容，支持GB 13000及Unicode的全部统一汉字，共收录汉字70244个。
4
5 GB 18030主要有以下特点：
6
7 与 UTF-8 相同，采用多字节编码，每个字可以由1个、2个或4个字节组成。
8 编码空间庞大，最多可定义161万个字元。
9 支持中国国内少数民族的文字，不需要动用造字区。
10 汉字收录范围包含繁体汉字以及日韩汉字。
11
12 本规格的初版是由中华人民共和国信息产业部电子工业标准化研究所起草，由国家质量技术监督局于2000年3月17日发布。现行版本为国家质量监督检验总局和中国国家标准化管理委员会于2005年11月8日发布，2006年5月1日实施。
13
14 此标准内的单字节编码部分、双字节编码部分，和四字节编码部分收录的中日韩统一表意文字扩展A区汉字，为强制性标准。其他部分则属于规模性标准。在中华人民共和国境内所有软件产品，都需要支持这个同时包含单字节、双字节和四字节编码的规格。
15
16 ฮขศํสำดฐด๚ย๋าณฮช 54936กฃ
+51
-0
src/test/resources/gb2312-sample.txt less more
0 GB18030ºÊ»›GBK£¨GBKºÊ»›GB2312°£
1
2 GBK全称《汉字内码扩展规范》（GBK即“国标”、“扩展”汉语拼音的第一个字母，英文名称：Chinese Internal Code Specification） ，中华人民共和国全国信息技术标准化技术委员会1995年12月1日制订，国家技术监督局标准化司、电子工业部科技与质量监督司1995年12月15日联合以技监标函1995 229号文件的形式，将它确定为技术规范指导性文件。这一版的GBK规范为1.0版。
3
4 GBK 向下与 GB 2312 编码兼容，向上支持 ISO 10646.1国际标准，是前者向后者过渡过程中的一个承上启下的产物。ISO 10646 是国际标准化组织 ISO 公布的一个编码标准，即 Universal Multilpe-Octet Coded Character Set（简称UCS），大陆译为《通用多八位编码字符集》，台湾译为《广用多八位元编码字元集》，它与 Unicode 组织的 Unicode 编码完全兼容。ISO 10646.1 是该标准的第一部分《体系结构与基本多文种平面》。我国 1993 年以 GB 13000.1 国家标准的形式予以认可（即 GB 13000.1 等同于 ISO 10646.1）。
5 GBK编码，是在GB2312-80标准基础上的内码扩展规范，使用了双字节编码方案，其编码范围从8140至FEFE（剔除xx7F），共23940个码位，共收录了21003个汉字，完全兼容GB2312-80标准，支持国际标准ISO/IEC10646-1和国家标准GB13000-1中的全部中日韩汉字，并包含了BIG5编码中的所有汉字。GBK编码方案于1995年10月制定， 1995年12月正式发布，中文版的WIN95、WIN98、WINDOWS NT以及WINDOWS 2000、WINDOWS XP、WIN 7等都支持GBK编码方案。
6
7 我们经常使用各种编码标准的汉字，编码到底是什么呢？所谓编码，是以固定的顺序排列字符，并以此做为记录、存贮、传递、交换的统一内部特征，这个字符排列顺序被称为“编码”。和中文字库有关的编码标准有：国标GB码、GBK码、港台BIG-5码等，不同编码的汉字字库都与汉字的应用有密切关系。
8 很多人在使用过程中，发现字不够用，因为大家使用的主要是GB编码字库，此编码标准只收录了6763个常用汉字，而GB字库以外大量汉字，只能通过方正女娲补字软件拼字或其它造字程序补字。尽管补出的汉字在字形上满足需要，但在字体风格、大小、结构方面难以协调统一，而采用手工贴图的方式补字，更不雅观。进而言之，如果用户建立信息系统，或需要查询新闻、出版内容时，靠补字是无法实现的。方正开发的GBK字库，将极大地缓解缺字现象。
9 从GB字库扩充到GBK字库，增加了1万4千多字。北大方正从1996年投入大量人力，开始做黑、宋、仿、楷GBK字库，并于1998年4月成为第一家通过国家权威部门组织的GBK字库鉴定的专业厂商。北大方正已将全部字体转换成GBK字库，共46款，其中18款字数达21003个，是拥有GBK字库款数最多的厂商。
10 ISO 10646 是一个包括世界上各种语言的书面形式以及附加符号的编码体系。其中的汉字部分称为“CJK 统一汉字”（C 指中国，J 指日本，K 指朝鲜）。而其中的中国部分，包括了源自中国大陆的 GB 2312、GB 12345、《现代汉语通用字表》等法定标准的汉字和符号，以及源自台湾的 CNS 11643 标准中第 1、2 字面（基本等同于 BIG-5 编码）、第 14 字面的汉字和符号。
11
12 ▒Я┬вий╩й
13 经实际测试和查阅文档，GBK是采用单双字节变长编码，英文使用单字节编码，完全兼容ASCII字符编码，中文部分采用双字节编码。
14 ×Ö»ã
15 GBK 规范收录了 ISO 10646.1 中的全部 CJK 汉字和符号，并有所补充。具体包括：
16 1. GB 2312 中的全部汉字、非汉字符号。
17 2. GB 13000.1 中的其他 CJK 汉字。以上合计 20902 个 GB 化汉字。
18 3. 《简化字总表》中未收入 GB 13000.1 的 52 个汉字。
19 4. 《康熙字典》及《辞海》中未收入 GB 13000.1 的 28 个部首及重要构件。
20 5. 13 ¸öºº×ֽṹ·û¡£
21 6. BIG-5 中未被 GB 2312 收入、但存在于 GB 13000.1 中的 139 个图形符号。
22 7. GB 12345 ิ๖ฒนตฤ 6 ธ๖ฦดา๔ท๛บลกฃ
23 8. ºº×Ö¡°¡ð¡±¡£
24 9. GB 12345 增补的 19 个竖排标点符号（GB 12345 较 GB 2312 增补竖排标点符号 29 个，其中 10 个未被 GB 13000.1 收入，故 GBK 亦不收）。
25 10. 从 GB 13000.1 的 CJK 兼容区挑选出的 21 个汉字。
26 11. GB 13000.1 收入的 31 个 IBM OS/2 专用符号。
27 12.未录入《新华字典》上的一些字，如“�|”的简体。
28 ∑÷≈‰º∞À≥–Ú
29 GBK 亦采用双字节表示，总体编码范围为 8140-FEFE，首字节在 81-FE 之间，尾字节在 40-FE 之间，剔除 xx7F 一条线。总计 23940 个码位，共收入 21886 个汉字和图形符号，其中汉字（包括部首和构件）21003 个，图形符号 883 个。
30 全部编码分为三大部分：
31 1. ºº×ÖÇø¡£°üÀ¨£º
32 a. GB 2312 汉字区。即 GBK/2: B0A1-F7FE。收录 GB 2312 汉字 6763 个，按原顺序排列。
33 b. GB 13000.1 À©³äºº×ÖÇø¡£°üÀ¨£º
34 (1) GBK/3: 8140-A0FE。收录 GB 13000.1 中的 CJK 汉字 6080 个。
35 (2) GBK/4: AA40-FEA0。收录 CJK 汉字和增补的汉字 8160 个。CJK 汉字在前，按 UCS 代码大小排列；增补的汉字（包括部首和构件）在后，按《康熙字典》的页码/字位排列。
36 2. м╪пн╥Ш╨егЬ║ё╟Эю╗ё╨
37 a. GB 2312 非汉字符号区。即 GBK/1: A1A1-A9FE。其中除 GB 2312 的符号外，还有 10 个小写罗马数字和 GB 12345 增补的符号。计符号 717 个。
38 b. GB 13000.1 扩充非汉字区。即 GBK/5: A840-A9A0。BIG-5 非汉字符号、结构符和“○”排列在此区。计符号 166 个。
39 3. 用户自定义区：分为(1)(2)(3)三个小区。
40 (1) AAA1-AFFE£¬Âëλ 564 ¸ö¡£
41 (2) F8A1-FEFE£¬Âëλ 658 ¸ö¡£
42 (3) A140-A7A0£¬Âëλ 672 ¸ö¡£
43 第(3)区尽管对用户开放，但限制使用，因为不排除未来在此区域增补新字符的可能性。
44 ×ÖÐÎ
45 GBK 对字形作了如下的规定：
46 1. 原则上与 GB 13000.1 G列（即源自中国大陆法定标准的汉字）下的字形/笔形保持一致。
47 2. 在 CJK 汉字认同规则的总框架内，对所有的 GBK 编码汉字实施“无重码正形”（“GB 化”）；即在不造成重码的前提下，尽量采用中国新字形。
48 3. 对于超出 CJK 汉字认同规则的、或认同规则尚未明确规定的汉字，在 GBK 码位上暂安放旧字形。这样，在许多情况下 GBK 收入了同一汉字的新旧两种字形。
49 4. 非汉字符号的字形，凡 GB 2312 已经包括的，与 GB 2312 保持一致；超出 GB 2312 的部分，与 GB 13000.1 保持一致。
50 5. 带声调的拼音字母取半角形式。⏎
+5
-0
src/test/resources/gbk-sample.txt less more
0 �h字�却a�U展����，�QGBK，全名�椤��h字�却a�U展����(GBK)》1.0版，由中�A人民共和��全��信息技�g���驶�技�g委�T��1995年12月1日制��，��家技�g�O督局���驶�司和��子工�I部科技�c�|量�O督司1995年12月15日��合以《技�g�撕�[1995]229��》文件的形式公布。 GBK共收��21886���h字和�D形符��，其中�h字（包括部首和��件）21003��，�D形符��883��。
1
2 GBK的K�椤��U展」的�h�Z拼音（kuòzhǎn）第一����母。英文全�QChinese Internal Code Extension Specification。
3
4 GBK 只�椤讣夹g����指��性文件」，不�凫���家���省���家�|量技�g�O督局於2000年3月17日推出了GB 18030-2000���剩�以取代GBK。GB 18030-2000除保留全部GBK���a�h字，在第二字��把能使用����再度�M行�U展，增加了大�s一百���h字及四字�����a空�g，但是��GBK作�樽蛹�全部保留。���⒖�GB 18030。
+39
-0
src/test/resources/iso2022jp.txt less more
0 ========================================================================
1 $B%3%s%=!<%k(J $B%"%W%j%1!<%7%g%s(J : universalchardet $B%W%m%8%'%/%H$N35MW(J
2 ========================================================================
3
4 $B$3$N(J universalchardet $B%"%W%j%1!<%7%g%s$O!"(JAppWizard $B$K$h$C$F:n@.$5$l$^$7$?!#(J
5
6 $B$3$N%U%!%$%k$K$O!"(Juniversalchardet $B%"%W%j%1!<%7%g%s$r9=@.$9$k3F%U%!%$%k$N(J
7 $BFbMF$N35N,$,5-=R$5$l$F$$$^$9!#(J
8
9
10 universalchardet.vcproj
11 $B$3$l$O!"%"%W%j%1!<%7%g%s(J $B%&%#%6!<%I$G@8@.$5$l$k(J VC++ $B%W%m%8%'%/%H$N%a%$%s$N(J
12 $B%W%m%8%'%/%H(J $B%U%!%$%k$G$9!#(J
13 $B%U%!%$%k$r@8@.$7$?(J Visual C++ $B$N%P!<%8%g%s>pJs$H!"%"%W%j%1!<%7%g%s(J
14 $B%&%#%6!<%I$GA*Br$7$?%W%i%C%H%U%)!<%`!"9=@.!"$*$h$S%W%m%8%'%/%H$N5!G=$K4X$9$k(J
15 $B>pJs$,5-=R$5$l$F$$$^$9!#(J
16
17 universalchardet.cpp
18 $B$3$l$O!"%a%$%s$N%"%W%j%1!<%7%g%s(J $B%=!<%9(J $B%U%!%$%k$G$9!#(J
19
20 /////////////////////////////////////////////////////////////////////////////
21 $B$=$NB>$NI8=`%U%!%$%k(J :
22
23 StdAfx.h, StdAfx.cpp
24 $B$3$l$i$N%U%!%$%k$O!"%3%s%Q%$%k:Q$_%X%C%@!<(J (PCH) $B%U%!%$%k(J
25 universalchardet.pch $B$H%W%j%3%s%Q%$%k:Q$_7?%U%!%$%k(J StdAfx.obj $B$r(J
26 $B%S%k%I$9$k$?$a$K;HMQ$7$^$9!#(J
27
28 /////////////////////////////////////////////////////////////////////////////
29 $B$=$NB>$N%a%b(J :
30
31 AppWizard $B$G$O(J "TODO:" $B%3%a%s%H$r;HMQ$7$F!"%f!<%6!<$,DI2C$^$?$O%+%9%?%^%$%:$9$k(J
32 $B%=!<%9ItJ,$r<($7$^$9!#(J
33
34 /////////////////////////////////////////////////////////////////////////////
35
36
37
38
+4
-0
src/test/resources/iso88598.txt less more
0 העברעאיש
1 פֿון װיקיפּעדיע
2
3 העברעיִש (עברית) איז אַ סעמיטישע שפּראַך. דאָס מאָדערנע העברעיִש (עברית ישראלית) איז באַזירט הױפּטזעכלעך אױף לשון קודש און פֿאַרצײַטישן אראמיש, מיט אַנטלײַונגען פֿון נישט־סעמיטישע שפּראַכן און עטלעכע ענדערונגען אין גראַמאַטיק. דאָס העברעיִש איז נײַ־געשאַפֿן געװאָרן װי אַ רײדשפּראַך אינעם סוף פֿון 19טן יאָרהונדערט ביז אָנהײב פֿונעם 20טן יאָרהונדערט דורך אליעזר בן יהודה. די אָפֿיציעלע הברה איז באַזירט אױף די וואקאלן פון דער ספרדישער טראַדיציע.
+39
-0
src/test/resources/shiftjis.txt less more
0 ========================================================================
1 コンソール アプリケーション : universalchardet プロジェクトの概要
2 ========================================================================
3
4 この universalchardet アプリケーションは、AppWizard によって作成されました。
5
6 このファイルには、universalchardet アプリケーションを構成する各ファイルの
7 内容の概略が記述されています。
8
9
10 universalchardet.vcproj
11 これは、アプリケーション ウィザードで生成される VC++ プロジェクトのメインの
12 プロジェクト ファイルです。
13 ファイルを生成した Visual C++ のバージョン情報と、アプリケーション
14 ウィザードで選択したプラットフォーム、構成、およびプロジェクトの機能に関する
15 î•ñ‚ª‹Lq‚³‚ê‚Ä‚¢‚Ü‚·B
16
17 universalchardet.cpp
18 これは、メインのアプリケーション ソース ファイルです。
19
20 /////////////////////////////////////////////////////////////////////////////
21 その他の標準ファイル :
22
23 StdAfx.h, StdAfx.cpp
24 これらのファイルは、コンパイル済みヘッダー (PCH) ファイル
25 universalchardet.pch とプリコンパイル済み型ファイル StdAfx.obj を
26 ビルドするために使用します。
27
28 /////////////////////////////////////////////////////////////////////////////
29 ‚»‚Ì‘¼‚̃ƒ‚ :
30
31 AppWizard では "TODO:" コメントを使用して、ユーザーが追加またはカスタマイズする
32 ソース部分を示します。
33
34 /////////////////////////////////////////////////////////////////////////////
35
36
37
38
+11
-0
src/test/resources/tis620.txt less more
0 ๏เป็นมนุษย์สุดประเสริฐเลิศคุณค่า
1 กว่าบรรดาฝูงสัตว์เดรัจฉนา
2 จงฟันฝ่าพัฒนาวิชาการ
3 อย่าล้างผลาญฤๅเข่นฆ่าบีฑาใคร
4 ไม่ถือโทษโกรธแช่งซัดฮึดฮัดด่า
5 หัดอภัยเหมือนกีฬาอัชฌาศัย
6 ปฏิบัติประพฤติกฎกำหนดใจ
7 พูดจาให้จ๊ะๆ จ๋าๆ น่าฟังเอยฯ๛
8 נסעףפץצקרש
9 อรหํ สมฺมา สมฺพุทฺโธ ภควา
10 พ๎ราห๎มณา อันว่าวิสุทธิพราหมณทั้งหลายนั้น
src/test/resources/utf16le.txt less more
Binary diff not shown
+39
-0
src/test/resources/utf8.txt less more
0 ========================================================================
1 コンソール アプリケーション : universalchardet プロジェクトの概要
2 ========================================================================
3
4 この universalchardet アプリケーションは、AppWizard によって作成されました。
5
6 このファイルには、universalchardet アプリケーションを構成する各ファイルの
7 内容の概略が記述されています。
8
9
10 universalchardet.vcproj
11 これは、アプリケーション ウィザードで生成される VC++ プロジェクトのメインの
12 プロジェクト ファイルです。
13 ファイルを生成した Visual C++ のバージョン情報と、アプリケーション
14 ウィザードで選択したプラットフォーム、構成、およびプロジェクトの機能に関する
15 情報が記述されています。
16
17 universalchardet.cpp
18 これは、メインのアプリケーション ソース ファイルです。
19
20 /////////////////////////////////////////////////////////////////////////////
21 その他の標準ファイル :
22
23 StdAfx.h, StdAfx.cpp
24 これらのファイルは、コンパイル済みヘッダー (PCH) ファイル
25 universalchardet.pch とプリコンパイル済み型ファイル StdAfx.obj を
26 ビルドするために使用します。
27
28 /////////////////////////////////////////////////////////////////////////////
29 その他のメモ :
30
31 AppWizard では "TODO:" コメントを使用して、ユーザーが追加またはカスタマイズする
32 ソース部分を示します。
33
34 /////////////////////////////////////////////////////////////////////////////
35
36
37
38
+39
-0
src/test/resources/utf8n-emoji.txt less more
0 ========================================================================
1 🍣コンソール アプリケーション : 🌏universalchardet プロジェクトの概要
2 ========================================================================
3
4 この universalchardet アプリケーションは、AppWizard によって作成されました。
5
6 このファイルには、universalchardet アプリケーションを構成する各ファイルの
7 内容の概略が記述されています。
8
9
10 universalchardet.vcproj
11 これは、アプリケーション ウィザードで生成される VC++ プロジェクトのメインの
12 プロジェクト ファイルです。
13 ファイルを生成した Visual C++ のバージョン情報と、アプリケーション
14 ウィザードで選択したプラットフォーム、構成、およびプロジェクトの機能に関する
15 情報が記述されています。
16
17 universalchardet.cpp
18 これは、メインのアプリケーション ソース ファイルです。
19
20 /////////////////////////////////////////////////////////////////////////////
21 その他の標準ファイル :
22
23 StdAfx.h, StdAfx.cpp
24 これらのファイルは、コンパイル済みヘッダー (PCH) ファイル
25 universalchardet.pch とプリコンパイル済み型ファイル StdAfx.obj を
26 ビルドするために使用します。
27
28 /////////////////////////////////////////////////////////////////////////////
29 その他のメモ :
30
31 AppWizard では "TODO:" コメントを使用して、ユーザーが追加またはカスタマイズする
32 ソース部分を示します。
33
34 /////////////////////////////////////////////////////////////////////////////
35
36
37
38
+39
-0
src/test/resources/utf8n.txt less more
0 ========================================================================
1 コンソール アプリケーション : universalchardet プロジェクトの概要
2 ========================================================================
3
4 この universalchardet アプリケーションは、AppWizard によって作成されました。
5
6 このファイルには、universalchardet アプリケーションを構成する各ファイルの
7 内容の概略が記述されています。
8
9
10 universalchardet.vcproj
11 これは、アプリケーション ウィザードで生成される VC++ プロジェクトのメインの
12 プロジェクト ファイルです。
13 ファイルを生成した Visual C++ のバージョン情報と、アプリケーション
14 ウィザードで選択したプラットフォーム、構成、およびプロジェクトの機能に関する
15 情報が記述されています。
16
17 universalchardet.cpp
18 これは、メインのアプリケーション ソース ファイルです。
19
20 /////////////////////////////////////////////////////////////////////////////
21 その他の標準ファイル :
22
23 StdAfx.h, StdAfx.cpp
24 これらのファイルは、コンパイル済みヘッダー (PCH) ファイル
25 universalchardet.pch とプリコンパイル済み型ファイル StdAfx.obj を
26 ビルドするために使用します。
27
28 /////////////////////////////////////////////////////////////////////////////
29 その他のメモ :
30
31 AppWizard では "TODO:" コメントを使用して、ユーザーが追加またはカスタマイズする
32 ソース部分を示します。
33
34 /////////////////////////////////////////////////////////////////////////////
35
36
37
38
+4
-0
src/test/resources/windows1255.txt less more
0 העברעאיש
1 פֿון װיקיפּעדיע
2
3 העברעיִש (עברית) איז אַ סעמיטישע שפּראַך. דאָס מאָדערנע העברעיִש (עברית ישראלית) איז באַזירט הױפּטזעכלעך אױף לשון קודש און פֿאַרצײַטישן אראמיש, מיט אַנטלײַונגען פֿון נישט־סעמיטישע שפּראַכן און עטלעכע ענדערונגען אין גראַמאַטיק. דאָס העברעיִש איז נײַ־געשאַפֿן געװאָרן װי אַ רײדשפּראַך אינעם סוף פֿון 19טן יאָרהונדערט ביז אָנהײב פֿונעם 20טן יאָרהונדערט דורך אליעזר בן יהודה. די אָפֿיציעלע הברה איז באַזירט אױף די וואקאלן פון דער ספרדישער טראַדיציע.