Codebase list shell-utils-clojure / 4b9bb6e
import shell-utils-clojure_1.0.2.orig.tar.xz Louis-Philippe Véronneau 3 years ago
21 changed file(s) with 811 addition(s) and 0 deletion(s). Raw diff Collapse all Expand all
+3
-0
.gitignore less more
0 .lein*
1 pom.xml
2 /target/⏎
+7
-0
.travis.yml less more
0 language: clojure
1 lein: 2.9.1
2 jdk:
3 - openjdk8
4 - openjdk11
5 notifications:
6 email: false
+3
-0
CHANGELOG.md less more
0 ## 1.0.0
1
2 * Initial release
+9
-0
CONTRIBUTING.md less more
0 # How to contribute
1
2 Third-party patches are essential for keeping puppet open-source projects
3 great. We want to keep it as easy as possible to contribute changes that
4 allow you to get the most out of our projects. There are a few guidelines
5 that we need contributors to follow so that we can have a chance of keeping on
6 top of things. For more info, see our canonical guide to contributing:
7
8 [https://github.com/puppetlabs/puppet/blob/master/CONTRIBUTING.md](https://github.com/puppetlabs/puppet/blob/master/CONTRIBUTING.md)
+201
-0
LICENSE less more
0 Apache License
1 Version 2.0, January 2004
2 http://www.apache.org/licenses/
3
4 TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
5
6 1. Definitions.
7
8 "License" shall mean the terms and conditions for use, reproduction,
9 and distribution as defined by Sections 1 through 9 of this document.
10
11 "Licensor" shall mean the copyright owner or entity authorized by
12 the copyright owner that is granting the License.
13
14 "Legal Entity" shall mean the union of the acting entity and all
15 other entities that control, are controlled by, or are under common
16 control with that entity. For the purposes of this definition,
17 "control" means (i) the power, direct or indirect, to cause the
18 direction or management of such entity, whether by contract or
19 otherwise, or (ii) ownership of fifty percent (50%) or more of the
20 outstanding shares, or (iii) beneficial ownership of such entity.
21
22 "You" (or "Your") shall mean an individual or Legal Entity
23 exercising permissions granted by this License.
24
25 "Source" form shall mean the preferred form for making modifications,
26 including but not limited to software source code, documentation
27 source, and configuration files.
28
29 "Object" form shall mean any form resulting from mechanical
30 transformation or translation of a Source form, including but
31 not limited to compiled object code, generated documentation,
32 and conversions to other media types.
33
34 "Work" shall mean the work of authorship, whether in Source or
35 Object form, made available under the License, as indicated by a
36 copyright notice that is included in or attached to the work
37 (an example is provided in the Appendix below).
38
39 "Derivative Works" shall mean any work, whether in Source or Object
40 form, that is based on (or derived from) the Work and for which the
41 editorial revisions, annotations, elaborations, or other modifications
42 represent, as a whole, an original work of authorship. For the purposes
43 of this License, Derivative Works shall not include works that remain
44 separable from, or merely link (or bind by name) to the interfaces of,
45 the Work and Derivative Works thereof.
46
47 "Contribution" shall mean any work of authorship, including
48 the original version of the Work and any modifications or additions
49 to that Work or Derivative Works thereof, that is intentionally
50 submitted to Licensor for inclusion in the Work by the copyright owner
51 or by an individual or Legal Entity authorized to submit on behalf of
52 the copyright owner. For the purposes of this definition, "submitted"
53 means any form of electronic, verbal, or written communication sent
54 to the Licensor or its representatives, including but not limited to
55 communication on electronic mailing lists, source code control systems,
56 and issue tracking systems that are managed by, or on behalf of, the
57 Licensor for the purpose of discussing and improving the Work, but
58 excluding communication that is conspicuously marked or otherwise
59 designated in writing by the copyright owner as "Not a Contribution."
60
61 "Contributor" shall mean Licensor and any individual or Legal Entity
62 on behalf of whom a Contribution has been received by Licensor and
63 subsequently incorporated within the Work.
64
65 2. Grant of Copyright License. Subject to the terms and conditions of
66 this License, each Contributor hereby grants to You a perpetual,
67 worldwide, non-exclusive, no-charge, royalty-free, irrevocable
68 copyright license to reproduce, prepare Derivative Works of,
69 publicly display, publicly perform, sublicense, and distribute the
70 Work and such Derivative Works in Source or Object form.
71
72 3. Grant of Patent License. Subject to the terms and conditions of
73 this License, each Contributor hereby grants to You a perpetual,
74 worldwide, non-exclusive, no-charge, royalty-free, irrevocable
75 (except as stated in this section) patent license to make, have made,
76 use, offer to sell, sell, import, and otherwise transfer the Work,
77 where such license applies only to those patent claims licensable
78 by such Contributor that are necessarily infringed by their
79 Contribution(s) alone or by combination of their Contribution(s)
80 with the Work to which such Contribution(s) was submitted. If You
81 institute patent litigation against any entity (including a
82 cross-claim or counterclaim in a lawsuit) alleging that the Work
83 or a Contribution incorporated within the Work constitutes direct
84 or contributory patent infringement, then any patent licenses
85 granted to You under this License for that Work shall terminate
86 as of the date such litigation is filed.
87
88 4. Redistribution. You may reproduce and distribute copies of the
89 Work or Derivative Works thereof in any medium, with or without
90 modifications, and in Source or Object form, provided that You
91 meet the following conditions:
92
93 (a) You must give any other recipients of the Work or
94 Derivative Works a copy of this License; and
95
96 (b) You must cause any modified files to carry prominent notices
97 stating that You changed the files; and
98
99 (c) You must retain, in the Source form of any Derivative Works
100 that You distribute, all copyright, patent, trademark, and
101 attribution notices from the Source form of the Work,
102 excluding those notices that do not pertain to any part of
103 the Derivative Works; and
104
105 (d) If the Work includes a "NOTICE" text file as part of its
106 distribution, then any Derivative Works that You distribute must
107 include a readable copy of the attribution notices contained
108 within such NOTICE file, excluding those notices that do not
109 pertain to any part of the Derivative Works, in at least one
110 of the following places: within a NOTICE text file distributed
111 as part of the Derivative Works; within the Source form or
112 documentation, if provided along with the Derivative Works; or,
113 within a display generated by the Derivative Works, if and
114 wherever such third-party notices normally appear. The contents
115 of the NOTICE file are for informational purposes only and
116 do not modify the License. You may add Your own attribution
117 notices within Derivative Works that You distribute, alongside
118 or as an addendum to the NOTICE text from the Work, provided
119 that such additional attribution notices cannot be construed
120 as modifying the License.
121
122 You may add Your own copyright statement to Your modifications and
123 may provide additional or different license terms and conditions
124 for use, reproduction, or distribution of Your modifications, or
125 for any such Derivative Works as a whole, provided Your use,
126 reproduction, and distribution of the Work otherwise complies with
127 the conditions stated in this License.
128
129 5. Submission of Contributions. Unless You explicitly state otherwise,
130 any Contribution intentionally submitted for inclusion in the Work
131 by You to the Licensor shall be under the terms and conditions of
132 this License, without any additional terms or conditions.
133 Notwithstanding the above, nothing herein shall supersede or modify
134 the terms of any separate license agreement you may have executed
135 with Licensor regarding such Contributions.
136
137 6. Trademarks. This License does not grant permission to use the trade
138 names, trademarks, service marks, or product names of the Licensor,
139 except as required for reasonable and customary use in describing the
140 origin of the Work and reproducing the content of the NOTICE file.
141
142 7. Disclaimer of Warranty. Unless required by applicable law or
143 agreed to in writing, Licensor provides the Work (and each
144 Contributor provides its Contributions) on an "AS IS" BASIS,
145 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
146 implied, including, without limitation, any warranties or conditions
147 of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
148 PARTICULAR PURPOSE. You are solely responsible for determining the
149 appropriateness of using or redistributing the Work and assume any
150 risks associated with Your exercise of permissions under this License.
151
152 8. Limitation of Liability. In no event and under no legal theory,
153 whether in tort (including negligence), contract, or otherwise,
154 unless required by applicable law (such as deliberate and grossly
155 negligent acts) or agreed to in writing, shall any Contributor be
156 liable to You for damages, including any direct, indirect, special,
157 incidental, or consequential damages of any character arising as a
158 result of this License or out of the use or inability to use the
159 Work (including but not limited to damages for loss of goodwill,
160 work stoppage, computer failure or malfunction, or any and all
161 other commercial damages or losses), even if such Contributor
162 has been advised of the possibility of such damages.
163
164 9. Accepting Warranty or Additional Liability. While redistributing
165 the Work or Derivative Works thereof, You may choose to offer,
166 and charge a fee for, acceptance of support, warranty, indemnity,
167 or other liability obligations and/or rights consistent with this
168 License. However, in accepting such obligations, You may act only
169 on Your own behalf and on Your sole responsibility, not on behalf
170 of any other Contributor, and only if You agree to indemnify,
171 defend, and hold each Contributor harmless for any liability
172 incurred by, or claims asserted against, such Contributor by reason
173 of your accepting any such warranty or additional liability.
174
175 END OF TERMS AND CONDITIONS
176
177 APPENDIX: How to apply the Apache License to your work.
178
179 To apply the Apache License to your work, attach the following
180 boilerplate notice, with the fields enclosed by brackets "{}"
181 replaced with your own identifying information. (Don't include
182 the brackets!) The text should be enclosed in the appropriate
183 comment syntax for the file format. We also recommend that a
184 file or class name and description of purpose be included on the
185 same "printed page" as the copyright notice for easier
186 identification within third-party archives.
187
188 Copyright {yyyy} {name of copyright owner}
189
190 Licensed under the Apache License, Version 2.0 (the "License");
191 you may not use this file except in compliance with the License.
192 You may obtain a copy of the License at
193
194 http://www.apache.org/licenses/LICENSE-2.0
195
196 Unless required by applicable law or agreed to in writing, software
197 distributed under the License is distributed on an "AS IS" BASIS,
198 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
199 See the License for the specific language governing permissions and
200 limitations under the License.
+28
-0
README.md less more
0 # puppetlabs/clj-shell-utils
1
2 A library for shell execution common to Puppet clojure projects.
3
4 ## Installation
5
6 Add the following dependency to your `project.clj` file:
7
8 [![Clojars Project](http://clojars.org/puppetlabs/clj-shell-utils/latest-version.svg)](http://clojars.org/puppetlabs/clj-shell-utils)
9
10 ## License
11
12 Copyright © 2019 Puppet Labs
13
14 See [LICENSE](LICENSE) file.
15
16 ## Support
17
18 Please log tickets and issues at our [JIRA tracker](https://tickets.puppetlabs.com/).
19
20 We use semantic version numbers for our releases, and recommend that users stay
21 as up-to-date as possible by upgrading to patch releases and minor releases as
22 they become available.
23
24 Bugfixes and ongoing development will occur in minor releases for the current
25 major version. Security fixes will be backported to a previous major version on
26 a best-effort basis, until the previous major version is no longer maintained.
27
+3
-0
dev-resources/puppetlabs/puppetserver/shell_utils_test/cat less more
0 #!/usr/bin/env sh
1
2 cat -⏎
+3
-0
dev-resources/puppetlabs/puppetserver/shell_utils_test/echo less more
0 #!/usr/bin/env sh
1
2 echo $@
+4
-0
dev-resources/puppetlabs/puppetserver/shell_utils_test/echo_and_warn less more
0 #!/usr/bin/env sh
1
2 echo "to out:" $@
3 echo "to err:" $@ 1>&2
+3
-0
dev-resources/puppetlabs/puppetserver/shell_utils_test/echo_foo_env_var less more
0 #!/usr/bin/env sh
1
2 echo $FOO
+3
-0
dev-resources/puppetlabs/puppetserver/shell_utils_test/false less more
0 #!/usr/bin/env sh
1
2 exit 1
+3
-0
dev-resources/puppetlabs/puppetserver/shell_utils_test/gen-output less more
0 #!/usr/bin/env sh
1
2 dd if=/dev/zero bs=$1 count=1 2>/dev/null
+7
-0
dev-resources/puppetlabs/puppetserver/shell_utils_test/list_env_var_names less more
0 #!/usr/bin/env sh
1
2 # -s supresses lines that don't have an equals sign in them (this prevents bar in FOO=foo\nbar from being interpretted as a environment key)
3 # -f selects which part of the split line to select
4 # -d specifies the delimiter
5 printenv | cut -s -f 1 -d "="
6
+3
-0
dev-resources/puppetlabs/puppetserver/shell_utils_test/num-args less more
0 #!/usr/bin/env sh
1
2 exit $#
+3
-0
dev-resources/puppetlabs/puppetserver/shell_utils_test/true less more
0 #!/usr/bin/env sh
1
2 exit 0
+3
-0
dev-resources/puppetlabs/puppetserver/shell_utils_test/warn less more
0 #!/usr/bin/env sh
1
2 echo $@ 1>&2
+40
-0
project.clj less more
0 (defproject puppetlabs/clj-shell-utils "1.0.2"
1 :description "Clojure shell execution utilities"
2
3 :min-lein-version "2.9.0"
4
5 :parent-project {:coords [puppetlabs/clj-parent "3.0.0"]
6 :inherit [:managed-dependencies]}
7
8 :pedantic? :abort
9
10 :test-paths ["test/unit"]
11
12 :plugins [[lein-project-version "0.1.0"]
13 [lein-parent "0.3.6"]]
14
15 :source-paths ["src/clj"]
16 :java-source-paths ["src/java"]
17
18 :dependencies [[org.clojure/clojure]
19 [prismatic/schema]
20 [org.apache.commons/commons-exec]
21 [commons-io]
22 [org.slf4j/log4j-over-slf4j]
23 [org.slf4j/slf4j-api]
24 [puppetlabs/trapperkeeper]
25 [puppetlabs/kitchensink]
26 [puppetlabs/i18n]]
27
28
29
30 :profiles { :test { :dependencies [[puppetlabs/trapperkeeper nil :classifier "test" :scope "test"]]}}
31
32 :deploy-repositories [["releases" {:url "https://clojars.org/repo"
33 :username :env/clojars_jenkins_username
34 :password :env/clojars_jenkins_password
35 :sign-releases false}]
36 ["snapshots" "http://nexus.delivery.puppetlabs.net/content/repositories/snapshots/"]]
37 )
38
39
+103
-0
src/clj/puppetlabs/puppetserver/shell_utils.clj less more
0 (ns puppetlabs.puppetserver.shell-utils
1 (:require [schema.core :as schema]
2 [clojure.java.io :as io]
3 [puppetlabs.kitchensink.core :as ks]
4 [clojure.string :as string]
5 [puppetlabs.i18n.core :as i18n])
6 (:import (com.puppetlabs.puppetserver ShellUtils ShellUtils$ExecutionOptions)
7 (java.io InputStream)
8 (org.apache.commons.io IOUtils)))
9
10 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
11 ;;; Schemas
12
13 (def ExecutionResult
14 "A map that contains the details of the result of executing a command."
15 {:exit-code schema/Int
16 :stderr schema/Str
17 :stdout schema/Str})
18
19 (def ExecutionResultStreamed
20 "A map that contains the details of the result of executing a command with
21 stdout as a stream."
22 {:exit-code schema/Int
23 :stderr schema/Str
24 :stdout InputStream})
25
26 (def ExecutionOptions
27 {(schema/optional-key :args) [schema/Str]
28 (schema/optional-key :env) (schema/maybe {schema/Str schema/Str})
29 (schema/optional-key :in) (schema/maybe InputStream)})
30
31 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
32 ;;; Internal
33
34 (def default-execution-options
35 {:args []
36 :env nil
37 :in nil})
38
39 (schema/defn ^:always-validate java-exe-options :- ShellUtils$ExecutionOptions
40 [{:keys [env in]} :- ExecutionOptions]
41 (let [exe-options (ShellUtils$ExecutionOptions.)]
42 (.setStdin exe-options in)
43 (when env
44 (.setEnv exe-options (ks/mapkeys name env)))
45 exe-options))
46
47 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
48 ;;; Public
49
50 (schema/defn ^:always-validate
51 validate-command!
52 "Checks the command string to ensure that it is an absolute path, executable
53 and that the file exists. An exception is thrown if any of those are not the
54 case."
55 [command :- schema/Str]
56 (let [command-file (io/as-file command)]
57 (cond
58 (not (.isAbsolute command-file))
59 (throw (IllegalArgumentException.
60 (i18n/trs "An absolute path is required, but ''{0}'' is not an absolute path" command)))
61 (not (.exists command-file))
62 (let [cmds (string/split command #" ")]
63 (if (and (> (count cmds) 1) (.exists (io/as-file (first cmds))))
64 (throw (IllegalArgumentException.
65 (i18n/trs "Command ''{0}'' appears to use command-line arguments, but this is not allowed." command)))
66 (throw (IllegalArgumentException.
67 (i18n/trs "The referenced command ''{0}'' does not exist" command)))))
68 (not (.canExecute command-file))
69 (throw (IllegalArgumentException.
70 (i18n/trs "The referenced command ''{0}'' is not executable" command))))))
71
72 (schema/defn ^:always-validate
73 execute-command-streamed :- ExecutionResultStreamed
74 "Execute the specified fully qualified command (string) and any included
75 command-arguments (vector of strings) and return the exit-code (integer),
76 and the contents of the stdout (stream) and stderr (string) for the command."
77 ([command :- schema/Str]
78 (execute-command-streamed command {}))
79 ([command :- schema/Str
80 opts :- ExecutionOptions]
81 (let [{:keys [args] :as opts} (merge default-execution-options opts)]
82 (validate-command! command)
83 (let [process (ShellUtils/executeCommand
84 command
85 (into-array String args)
86 (java-exe-options opts))]
87 {:exit-code (.getExitCode process)
88 :stderr (.getError process)
89 :stdout (.getOutputAsStream process)}))))
90
91 (schema/defn ^:always-validate
92 execute-command :- ExecutionResult
93 "Execute the specified fully qualified command (string) and any included
94 command-arguments (vector of strings) and return the exit-code (integer),
95 and the contents of the stdout (string) and stderr (string) for the command."
96 ([command :- schema/Str]
97 (execute-command command {}))
98 ([command :- schema/Str
99 opts :- ExecutionOptions]
100 (let [result (execute-command-streamed command opts)]
101 (update-in result [:stdout]
102 (fn [stdout] (IOUtils/toString stdout "UTF-8"))))))
+33
-0
src/java/com/puppetlabs/puppetserver/ExecutionResult.java less more
0 package com.puppetlabs.puppetserver;
1
2 import java.io.InputStream;
3 import java.io.IOException;
4 import org.apache.commons.io.IOUtils;
5
6 public class ExecutionResult {
7 private final InputStream output;
8 private final String error;
9 private final int exitCode;
10
11 public ExecutionResult(InputStream output, String error, int exitCode) {
12 this.output = output;
13 this.error = error;
14 this.exitCode = exitCode;
15 }
16
17 public int getExitCode() {
18 return exitCode;
19 }
20
21 public String getOutput() throws IOException {
22 return IOUtils.toString(output, "UTF-8");
23 }
24
25 public InputStream getOutputAsStream() {
26 return output;
27 }
28
29 public String getError() {
30 return error;
31 }
32 }
+220
-0
src/java/com/puppetlabs/puppetserver/ShellUtils.java less more
0 package com.puppetlabs.puppetserver;
1
2 import org.apache.commons.exec.DefaultExecutor;
3 import org.apache.commons.exec.PumpStreamHandler;
4 import org.apache.commons.exec.CommandLine;
5 import org.apache.commons.io.output.TeeOutputStream;
6 import org.slf4j.Logger;
7 import org.slf4j.LoggerFactory;
8
9 import java.io.ByteArrayOutputStream;
10 import java.io.ByteArrayInputStream;
11 import java.io.IOException;
12 import java.io.InputStream;
13 import java.util.Map;
14
15 public class ShellUtils {
16
17 public static class ExecutionOptions {
18 private boolean combineStdoutStderr = false;
19 private Map<String, String> env = null;
20 private InputStream stdin = null;
21
22 public boolean getCombineStdoutStderr() {
23 return combineStdoutStderr;
24 }
25
26 public void setCombineStdoutStderr(boolean combineStdoutStderr) {
27 this.combineStdoutStderr = combineStdoutStderr;
28 }
29
30 public InputStream getStdin() {
31 return stdin;
32 }
33
34 public void setStdin(InputStream stdin) {
35 this.stdin = stdin;
36 }
37
38 public Map<String, String> getEnv() {
39 return env;
40 }
41
42 public void setEnv(Map<String, String> env) {
43 this.env = env;
44 }
45 }
46
47 private static final Logger log = LoggerFactory.getLogger(ShellUtils.class);
48
49 /**
50 * Takes a prepared CommandLine instance and executes it using some sane
51 * defaults and a DefaultExecutor. Also makes a delicious pancake.
52 *
53 * @param commandLine CommandLine instance to execute
54 * @param options optional object [ExecutionOptions] to control behavior; may be null.
55 * @return An ExecutionResult with output[String], error[String], and
56 * the exit code[Integer] of the process
57 *
58 * @throws InterruptedException
59 * @throws IOException
60 */
61 private static ExecutionResult executeExecutor(CommandLine commandLine,
62 ExecutionOptions options)
63 throws InterruptedException, IOException {
64 if (options == null) {
65 options = new ExecutionOptions();
66 }
67 DefaultExecutor executor = new DefaultExecutor();
68 ByteArrayOutputStream errStream = new ByteArrayOutputStream();
69 // TODO the nice thing here would be to set up a piped stream
70 // arrangement like:
71 // PipedOutputStream stdoutOutputStream = new PipedOutputStream();
72 // PipedInputStream stdoutInputStream = new PipedInputStream(stdoutOutputStream);
73 // but this requires that the input stream be read on a different thread
74 // than this one. this is currently out of scope.
75 ByteArrayOutputStream stdoutOutputStream = new ByteArrayOutputStream();
76 PumpStreamHandler streamHandler;
77 if (options.getCombineStdoutStderr()) {
78 log.debug("Combining STDOUT/STDERR for external command '" + commandLine.toString() + "'");
79 streamHandler = new PumpStreamHandler(
80 stdoutOutputStream,
81 new TeeOutputStream(stdoutOutputStream, errStream),
82 options.getStdin());
83 } else {
84 streamHandler = new PumpStreamHandler(stdoutOutputStream, errStream,
85 options.getStdin());
86 }
87
88 // Don't throw exception on non-zero exit code
89 executor.setExitValues(null);
90
91 // Set up the handlers
92 executor.setStreamHandler(streamHandler);
93
94 Integer exitCode = executor.execute(commandLine, options.getEnv());
95
96 ByteArrayInputStream stdoutInputStream = new ByteArrayInputStream(stdoutOutputStream.toByteArray());
97 String stdErr = errStream.toString();
98
99 if ( ! stdErr.isEmpty() ) {
100 log.warn("Executed an external process which logged to STDERR: " + stdErr);
101 }
102
103 return new ExecutionResult(stdoutInputStream, stdErr, exitCode);
104 }
105
106 /**
107 * Executes the given command in a separate process.
108 *
109 * @param commandLine the command to execute.
110 * @param options optional object [ExecutionOptions] to control behavior; may be null.
111 *
112 * @return An ExecutionResult with output[String], error[String], and
113 * the exit code[Integer] of the process
114 *
115 * @throws InterruptedException
116 */
117 protected static ExecutionResult executeCommand(CommandLine commandLine,
118 ExecutionOptions options)
119 throws InterruptedException {
120 try {
121 return executeExecutor(commandLine, options);
122 } catch (IOException e) {
123 // this nonsense is due to a weird edge-case incompatibility between JDK8
124 // and apache commons-exec. See SERVER-1116; hopefully we can remove this
125 // conditional once that is resolved.
126 if (e.getMessage() == "Stream closed") {
127 log.warn("An error occurred while executing the command '" + commandLine.getExecutable() +
128 ". The most likely culprit is that you are on JDK8, " +
129 "and we executed an external process with data on its STDIN that was not " +
130 "consumed by the process. Please make sure the command above processes STDIN " +
131 "correctly. For more information, see " +
132 "https://tickets.puppetlabs.com/browse/SERVER-1116 . If you do not believe " +
133 "that this is the root cause of this error message, please file a bug at " +
134 "https://tickets.puppetlabs.com/browse/SERVER.");
135 }
136 throw new IllegalStateException(
137 "Exception while executing '" + commandLine.getExecutable() + "': " + e.getMessage(),
138 e);
139 }
140 }
141
142 /**
143 * Executes the given command in a separate process.
144 *
145 * @param command the command [String] to execute. arguments can be
146 * included in the string.
147 * @return An ExecutionResult with output[String], error[String], and
148 * the exit code[Integer] of the process
149 *
150 * @throws InterruptedException
151 * @throws IOException
152 */
153 public static ExecutionResult executeCommand(String command)
154 throws InterruptedException, IOException {
155 CommandLine commandLine = CommandLine.parse(command);
156
157 return executeCommand(commandLine, null);
158 }
159
160 /**
161 * Executes the given command in a separate process.
162 *
163 *
164 * @param command the command [String] to execute. arguments can be
165 * included in the string.
166 * @param options optional object [ExecutionOptions] to control behavior; may be null.
167 * @return An ExecutionResult with output[String], error[String], and
168 * the exit code[Integer] of the process
169 *
170 * @throws InterruptedException
171 * @throws IOException
172 */
173 public static ExecutionResult executeCommand(String command, ExecutionOptions options)
174 throws InterruptedException, IOException {
175 CommandLine commandLine = CommandLine.parse(command);
176
177 return executeCommand(commandLine, options);
178 }
179
180 /**
181 * Executes the given command in a separate process.
182 *
183 * @param command the command [String] to execute.
184 * @param arguments arguments [Array of Strings] to add to the command being executed
185 * @return An ExecutionResult with output[String], error[String], and
186 * the exit code[Integer] of the process
187 *
188 * @throws InterruptedException
189 * @throws IOException
190 */
191 public static ExecutionResult executeCommand(String command, String[] arguments)
192 throws InterruptedException, IOException {
193 return executeCommand(command, arguments, null);
194 }
195
196 /**
197 * Executes the given command in a separate process.
198 *
199 * @param command the command [String] to execute.
200 * @param arguments arguments [Array of Strings] to add to the command being executed
201 * @param options optional object [ExecutionOptions] to control behavior; may be null.
202 *
203 * @return An ExecutionResult with output[String], error[String], and
204 * the exit code[Integer] of the process
205 *
206 * @throws InterruptedException
207 * @throws IOException
208 */
209 public static ExecutionResult executeCommand(String command, String[] arguments,
210 ExecutionOptions options)
211 throws InterruptedException, IOException {
212 CommandLine commandLine = new CommandLine(command);
213 commandLine.addArguments(arguments, false);
214
215 return executeCommand(commandLine, options);
216 }
217
218
219 }
+129
-0
test/unit/puppetlabs/puppetserver/shell_utils_test.clj less more
0 (ns puppetlabs.puppetserver.shell-utils-test
1 (:require [clojure.test :refer :all]
2 [puppetlabs.puppetserver.shell-utils :as sh-utils]
3 [puppetlabs.kitchensink.core :as ks]
4 [puppetlabs.trapperkeeper.testutils.logging :as logging]
5 [clojure.string :as str]
6 [clojure.set :as set])
7 (:import (java.io ByteArrayInputStream)
8 (com.puppetlabs.puppetserver ShellUtils ShellUtils$ExecutionOptions)))
9
10 (def test-resources
11 (ks/absolute-path
12 "./dev-resources/puppetlabs/puppetserver/shell_utils_test"))
13
14 (defn script-path
15 [script-name]
16 (str test-resources "/" script-name))
17
18 (defn parse-env-output
19 [env-output]
20 (set (str/split-lines env-output)))
21
22 (deftest returns-the-exit-code
23 (testing "true should return 0"
24 (is (zero? (:exit-code (sh-utils/execute-command (script-path "true"))))))
25 (testing "false should return 1"
26 (is (= 1 (:exit-code (sh-utils/execute-command (script-path "false")))))))
27
28 (deftest returns-stdout-correctly
29 (testing "echo should add content to stdout"
30 (is (= "foo\n" (:stdout (sh-utils/execute-command
31 (script-path "echo")
32 {:args ["foo"]}))))))
33
34 (deftest returns-stderr-correctly
35 (testing "echo can add content to stderr as well"
36 (logging/with-test-logging
37 (is (= "bar\n" (:stderr (sh-utils/execute-command
38 (script-path "warn")
39 {:args ["bar"]})))))))
40
41 (deftest combines-stderr-and-stdout-correctly
42 (logging/with-test-logging
43 (let [options (ShellUtils$ExecutionOptions.)
44 _ (.setCombineStdoutStderr options true)
45 results (ShellUtils/executeCommand (str (script-path "echo_and_warn")
46 " baz")
47 options)]
48 (testing "combined info echoed to stdout and stderr captured as output"
49 (let [output (.getOutput results)]
50 ;; Allow stdout and stderr messages to come in either order since
51 ;; the order in which they are read from the different stream
52 ;; consuming threads is not reliable.
53 (is (or (= "to out: baz\nto err: baz\n" output)
54 (= "to err: baz\nto out: baz\n" output))
55 (format "Output produced, '%s', did not match expected output"
56 output))))
57 (testing "only info echoed to stderr captured as error"
58 (is (= "to err: baz\n" (.getError results))))
59 (testing "only stderr info (and not stdout info) is logged"
60 (is (logged?
61 "Executed an external process which logged to STDERR: to err: baz\n"
62 :warn))))))
63
64 (deftest pass-args-correctly
65 (testing "passes the expected number of args to cmd"
66 (is (= 5 (:exit-code (sh-utils/execute-command
67 (script-path "num-args")
68 {:args ["a" "b" "c" "d" "e"]}))))))
69
70 (deftest inherits-env-correctly
71 (testing "inherits environment variables if not specified"
72 (let [env-output (:stdout (sh-utils/execute-command
73 (script-path "list_env_var_names")))
74 env (parse-env-output env-output)]
75 (is (< 3 (count env))
76 (str "Expected at least 3 environment variables, got:\n" env-output))
77 (is (contains? env "PATH"))
78 (is (contains? env "PWD"))
79 (is (contains? env "HOME")))))
80
81 (deftest sets-env-correctly
82 (testing "sets environment variables correctly"
83 (is (= "foo\n" (:stdout (sh-utils/execute-command
84 (script-path "echo_foo_env_var")
85 {:env {"FOO" "foo"}}))))
86
87 (let [env-output (:stdout (sh-utils/execute-command
88 (script-path "list_env_var_names")
89 {:env {"FOO" "foo\nbar"}}))
90 env (parse-env-output env-output)
91 ;; it seems that the JVM always includes a PWD env var, no
92 ;; matter what, and in certain terminals it may also include a few
93 ;; other vars, so we are writing the test to be tolerant of that.
94 expected-keys #{"FOO" "PWD" "_" "SHLVL"}
95 extra-keys (set/difference env expected-keys)]
96 (is (empty? extra-keys)
97 (str "Found unexpected environment variables:" extra-keys)))))
98
99 (deftest pass-stdin-correctly
100 (testing "passes stdin stream to command"
101 (is (= "foo" (:stdout (sh-utils/execute-command
102 (script-path "cat")
103 {:in (ByteArrayInputStream.
104 (.getBytes "foo" "UTF-8"))}))))))
105
106 (deftest throws-exception-for-non-absolute-path
107 (testing "Commands must be given using absolute paths"
108 (is (thrown? IllegalArgumentException
109 (sh-utils/execute-command "echo")))))
110
111 (deftest throws-exception-for-non-existent-file
112 (testing "The given command must exist"
113 (is (thrown-with-msg? IllegalArgumentException
114 #"command '/usr/bin/footest' does not exist"
115 (sh-utils/execute-command "/usr/bin/footest")))))
116
117 (deftest throws-reasonable-error-for-arguments-in-command
118 (testing "A meaningful error is raised if arguments are added to the command"
119 (is (thrown-with-msg? IllegalArgumentException
120 #"appears to use command-line arguments, but this is not allowed"
121 (sh-utils/execute-command
122 (str (script-path "echo") " foo"))))))
123
124 (deftest can-read-more-than-the-pipe-buffer
125 (testing "Doesn't deadlock when reading more than the pipe can hold"
126 (is (= 128000 (count (:stdout (sh-utils/execute-command
127 (script-path "gen-output")
128 {:args ["128000"]})))))))