New Upstream Release - escapevelocity-java
Ready changes
Summary
Merged new upstream version: 1.1 (was: 0.9.1).
Resulting package
Built on 2023-02-09T06:36 (took 6m35s)
The resulting binary packages can be installed (if you have the apt repository enabled) by running one of:
apt install -t fresh-releases libescapevelocity-java
Lintian Result
Diff
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
new file mode 100644
index 0000000..a9663e7
--- /dev/null
+++ b/.github/workflows/ci.yml
@@ -0,0 +1,43 @@
+name: CI
+
+on:
+ push:
+ branches:
+ - main
+ pull_request:
+ branches:
+ - main
+
+jobs:
+ test:
+ name: "JDK ${{ matrix.java }}"
+ strategy:
+ matrix:
+ java: [ 8, 11, 15 ]
+ runs-on: ubuntu-latest
+ steps:
+ # Cancel any previous runs for the same branch that are still running.
+ - name: 'Cancel previous runs'
+ uses: styfle/cancel-workflow-action@0.8.0
+ with:
+ access_token: ${{ github.token }}
+ - name: 'Check out repository'
+ uses: actions/checkout@v2
+ - name: 'Cache local Maven repository'
+ uses: actions/cache@v2.1.4
+ with:
+ path: ~/.m2/repository
+ key: maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ maven-
+ - name: 'Set up JDK ${{ matrix.java }}'
+ uses: actions/setup-java@v2
+ with:
+ java-version: ${{ matrix.java }}
+ distribution: 'zulu'
+ - name: 'Install'
+ shell: bash
+ run: mvn -B install -U -DskipTests=true
+ - name: 'Test'
+ shell: bash
+ run: mvn -B verify -U -Dmaven.javadoc.skip=true
diff --git a/README.md b/README.md
index e716ecf..a7b6bb8 100644
--- a/README.md
+++ b/README.md
@@ -14,7 +14,6 @@ the exact same string that Velocity produces. If not, that is a bug.
EscapeVelocity has no facilities for HTML escaping and it is not appropriate for producing
HTML output that might include portions of untrusted input.
-
## Motivation
Velocity has a convenient templating language. It is easy to read, and it has widespread support
@@ -22,10 +21,13 @@ from tools such as editors and coding websites. However, *using* Velocity can pr
Its use to generate Java code in the [AutoValue][AutoValue] annotation processor required many
[workarounds][VelocityHacks]. The way it dynamically loads classes as part of its standard operation
makes it hard to [shade](https://maven.apache.org/plugins/maven-shade-plugin/) it, which in the case
-of AutoValue led to interference if Velocity was used elsewhere in a project.
+of AutoValue led to interference if Velocity was used elsewhere in a project. Velocity also has a
+large and complex API, and has introduced several incompatible changes over the years.
-EscapeVelocity has a simple API that does not involve any class-loading or other sources of
-problems. It and its dependencies can be shaded with no difficulty.
+EscapeVelocity has a
+[simple API](https://javadoc.io/doc/com.google.escapevelocity/escapevelocity/latest/index.html)
+that does not involve any class-loading or other sources of problems. It and its
+dependencies can be shaded with no difficulty. We take care to avoid incompatible changes.
## Loading a template
@@ -42,8 +44,7 @@ InputStream in = getClass().getResourceAsStream("foo.vm");
if (in == null) {
throw new IllegalArgumentException("Could not find resource foo.vm");
}
-Reader reader = new BufferedReader(new InputStreamReader(in));
-Template template = Template.parseFrom(reader);
+Template template = Template.parseFrom(new InputStreamReader(in));
```
## Expanding a template
@@ -93,7 +94,7 @@ EscapeVelocity supports most of the reference types described in the
### Variables
A variable has an ASCII name that starts with a letter (a-z or A-Z) and where any other characters
-are also letters or digits or hyphens (-) or underscores (_). A variable reference can be written
+are also letters or digits or hyphens (-) or underscores (`_`). A variable reference can be written
as `$foo` or as `${foo}`. The value of a variable can be of any Java type. If the value `v` of
variable `foo` is not a String then the result of `$foo` in a template will be `String.valueOf(v)`.
Variables must be defined before they are referenced; otherwise an `EvaluationException` will be
@@ -123,6 +124,9 @@ reference, you can use braces like this: `${purchase}.Total`. If, after a proper
have a further period, you can put braces around the reference like this:
`${purchase.Total}.nonProperty`.
+As a special case, if `$purchase` is a Java `Map`, `$purchase.Total` is the result of calling
+`get("Total")` on the `Map`.
+
### Methods
If a reference looks like `$purchase.addItem("scones", 23)` then the value of the `$purchase`
@@ -135,6 +139,16 @@ Properties are in fact a special case of methods: instead of writing `$purchase.
write `$purchase.getTotal()`. Braces can be used to make the method invocation explicit
(`${purchase.getTotal()}`) or to prevent method invocation (`${purchase}.getTotal()`).
+If the object that the method is being called on is an instance of `java.lang.Class`, then the
+method can be one of the methods of `java.lang.Class`, _or_ it can be a static method in the
+class in question. For example if `$Objects` is `java.util.Objects.class`, then
+`$Objects.equals($a, $b)` will invoke the static method
+[`java.util.Objects.equals`](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/Objects.html#equals(java.lang.Object,java.lang.Object))
+with the given parameters.
+
+A method parameter can be `null` to indicate a null value. For example
+`$Objects.equals(null, null)` would evaluate to `true`, given the above definition of `$Objects`.
+
### Indexing
If a reference looks like `$indexme[$i]` then the value of the `$indexme` variable must be a Java
@@ -144,6 +158,9 @@ be the result of `List.get(int)` for that list and that integer. Or, `$indexme`
and the reference would be the result of `Map.get(Object)` for the object `$i`. In general,
`$indexme[$i]` is equivalent to `$indexme.get($i)`.
+For lists specifically, the index can be negative, and then it counts from the end of the list.
+For example `$list[-1]` is the last element of `$list`.
+
Unlike Velocity, EscapeVelocity does not allow `$indexme` to be a Java array.
### Undefined references
@@ -153,6 +170,14 @@ set in the template, then referencing it will provoke an `EvaluationException`.
a special case for `#if`: if you write `#if ($var)` then it is allowed for `$var` not to be defined,
and it is treated as false.
+### Null references
+
+A reference can produce a null value, for example `$foo` if the input `Map` has an entry for `"foo"`
+with a null value, or `$indexme[$i]` if `$indexme` is a `List` that has a null element at index
+`$i`. If you try to insert a null reference into the output of a template then you will get an
+exception. If you use `$!` instead of `$`, like `$!foo` or `$!indexme[$i]`, then a null reference
+will instead produce nothing in the output.
+
### Setting properties and indexes: not supported
Unlke Velocity, EscapeVelocity does not allow `#set` assignments with properties or indexes:
@@ -168,18 +193,33 @@ In certain contexts, such as the `#set` directive we have just seen or certain o
EscapeVelocity can evaluate expressions. An expression can be any of these:
* A reference, of the kind we have just seen. The value is the value of the reference.
-* A string literal enclosed in double quotes, like `"this"`. A string literal must appear on
- one line. EscapeVelocity does not support the characters `$` or `\\` in a string literal.
+* A string literal, as described below.
* An integer literal such as `23` or `-100`. EscapeVelocity does not support floating-point
literals.
* A Boolean literal, `true` or `false`.
+* A list literal, as described below.
* Simpler expressions joined together with operators that have the same meaning as in Java:
`!`, `==`, `!=`, `<`, `<=`, `>`, `>=`, `&&`, `||`, `+`, `-`, `*`, `/`, `%`. The operators have the
same precedence as in Java.
* A simpler expression in parentheses, for example `(2 + 3)`.
-Velocity supports string literals with single quotes, like `'this`' and also references within
-strings, like `"a $reference in a string"`, but EscapeVelocity does not.
+### String literals
+
+There are two forms of string literals that can appear in expressions. The simpler form is
+surrounded with single quotes (`'...'`) and represents a string containing everything between those
+quotes. The other form is surrounded with double quotes (`"..."`) and again represents a string
+containing everything between the quotes, but this time the text can contain references like
+`$purchase.Total` and directives like `#if ($condition) yes #end`.
+
+String literals can span more than one line.
+
+### List literals
+
+There are two forms of list literals that can appear in expressions. An explicit list
+such as `[]`, `[23]`, or `["a", "b"]` evaluates to a Java `List` containing those values.
+A range such as `[0..$i]` or `[$from .. $to]` evaluates to a Java `List` containing the
+integer values from the first number to the second number, inclusive. If the second number
+is less than the first, the list values decrease.
## Directives
@@ -248,20 +288,33 @@ If `$allProducts` is a `List` containing the strings `oranges` and `lemons` then
When the `#foreach` completes, the loop variable (`$product` in the example) goes back to whatever
value it had before, or to being undefined if it was undefined before.
-Within the `#foreach`, a special variable `$foreach` is defined, such that you can write
-`$foreach.hasNext`, which will be true if there are more values after this one or false if this
-is the last value. For example:
+Within the `#foreach`, the special variable `$foreach` is defined.
+
+`$foreach.hasNext` will be true if there are more values after this one or false if this
+is the last value. `$foreach.index` will be the index of the iteration, starting at 0. For example:
```
-#foreach ($product in $allProducts)${product}#if ($foreach.hasNext), #end#end
+#foreach ($product in $allProducts)${foreach.index}: ${product}#if ($foreach.hasNext), #end#end
```
-This would produce the output `oranges, lemons` for the list above. (The example is scrunched up
+This would produce the output `0: oranges, 1: lemons` for the list above. (The example is scrunched up
to avoid introducing extraneous spaces, as described in the [section](#spaces) on spaces
below.)
-Velocity gives the `$foreach` variable other properties (`index` and `count`) but EscapeVelocity
-does not.
+`$foreach.first` and `$foreach.last` are true for the first and last iteration, respectively, and false
+for other iterations. So `$foreach.last` is the negation of `$foreach.hasNext`.
+
+`$foreach.count` is one more than `$foreach.index`.
+
+The `#foreach` directive is often used with list literals:
+
+```
+#foreach ($i in [1..$n])
+ #foreach ($j in ["a", "b", "c"])
+ $someObject.someMethod($i, $j)
+ #end
+#end
+```
### Macros
@@ -334,11 +387,11 @@ For this to work, you will need to tell EscapeVelocity how to find "resources" s
```
ResourceOpener resourceOpener = resourceName -> {
- InputStream inputStream = getClass().getResource(resourceName);
+ InputStream inputStream = getClass().getResource(resourceName).openStream();
if (inputStream == null) {
throw new IOException("Unknown resource: " + resourceName);
}
- return new BufferedReader(InputStreamReader(inputStream, StandardCharsets.UTF_8));
+ return new InputStreamReader(inputStream, StandardCharsets.UTF_8);
};
Template template = Template.parseFrom("foo.vm", resourceOpener);
```
@@ -346,6 +399,11 @@ Template template = Template.parseFrom("foo.vm", resourceOpener);
In this case, the `resourceOpener` is used to find the main template `foo.vm`, as well as any
templates it may reference in `#parse` directives.
+A `#parse` directive only reads and parses the named template (`macros.vm` in the example)
+when the containing template (`foo.vm`) is evaluated (`template.evaluate(vars)`). The result
+is cached, so if you do `template.evaluate(vars)` a second time it will use the already-parsed
+`macros.vm` from the first time.
+
## <a name="spaces"></a> Spaces
For the most part, spaces and newlines in the template are preserved exactly in the output.
@@ -376,4 +434,4 @@ post-process it. For example, if it is Java code, you could use a formatter such
worry about extraneous spaces.
[VelocityHacks]: https://github.com/google/auto/blob/ca2384d5ad15a0c761b940384083cf5c50c6e839/value/src/main/java/com/google/auto/value/processor/TemplateVars.java#L54
-[AutoValue]: https://github.com/google/auto/tree/master/value
+[AutoValue]: https://github.com/google/auto/tree/main/value
diff --git a/debian/changelog b/debian/changelog
index 43ab453..7a55907 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+escapevelocity-java (1.1-1) UNRELEASED; urgency=low
+
+ * New upstream release.
+
+ -- Debian Janitor <janitor@jelmer.uk> Thu, 09 Feb 2023 06:30:40 -0000
+
escapevelocity-java (0.9.1-2) unstable; urgency=medium
* Fix VCS in d/control
diff --git a/debian/patches/verbose-build.patch b/debian/patches/verbose-build.patch
index 351b57d..4eadeea 100644
--- a/debian/patches/verbose-build.patch
+++ b/debian/patches/verbose-build.patch
@@ -2,8 +2,10 @@ Description: Enable verbose build messages
Author: Olek Wojnar <olek@debian.org>
Last-Update: 2020-06-03
---- a/pom.xml
-+++ b/pom.xml
+Index: escapevelocity-java.git/pom.xml
+===================================================================
+--- escapevelocity-java.git.orig/pom.xml
++++ escapevelocity-java.git/pom.xml
@@ -88,6 +88,7 @@
<source>1.8</source>
<target>1.8</target>
diff --git a/pom.xml b/pom.xml
index 7f0e0a5..8f4853b 100644
--- a/pom.xml
+++ b/pom.xml
@@ -24,7 +24,7 @@
<groupId>com.google.escapevelocity</groupId>
<artifactId>escapevelocity</artifactId>
- <version>0.9.1</version>
+ <version>1.1</version>
<name>EscapeVelocity</name>
<description>
A reimplementation of a subset of the Apache Velocity templating system.
@@ -49,7 +49,7 @@
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
- <version>23.5-jre</version>
+ <version>31.1-jre</version>
</dependency>
<!-- test dependencies -->
<dependency>
@@ -61,19 +61,19 @@
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava-testlib</artifactId>
- <version>23.5-jre</version>
+ <version>31.1-jre</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
- <version>4.12</version>
+ <version>4.13.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.truth</groupId>
<artifactId>truth</artifactId>
- <version>0.44</version>
+ <version>1.1.3</version>
<scope>test</scope>
</dependency>
</dependencies>
@@ -83,7 +83,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <version>3.7.0</version>
+ <version>3.10.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
@@ -95,12 +95,12 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
- <version>3.0.2</version>
+ <version>3.3.0</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-invoker-plugin</artifactId>
- <version>3.0.1</version>
+ <version>3.3.0</version>
</plugin>
</plugins>
</build>
diff --git a/src/main/java/com/google/escapevelocity/BreakException.java b/src/main/java/com/google/escapevelocity/BreakException.java
new file mode 100644
index 0000000..81448e4
--- /dev/null
+++ b/src/main/java/com/google/escapevelocity/BreakException.java
@@ -0,0 +1,36 @@
+/*
+ * Copyright (C) 2023 Google, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.google.escapevelocity;
+
+/** Exception thrown when a {@code #break} is encountered. */
+final class BreakException extends RuntimeException {
+ private final boolean forEachScope;
+
+ /**
+ * Constructs a new {@code BreakException}.
+ *
+ * @param forEachScope true if this is {@code #break($foreach)}. Then if we are not actually
+ * inside a {@code #foreach} it is an error.
+ */
+ BreakException(String message, boolean forEachScope) {
+ super(message);
+ this.forEachScope = forEachScope;
+ }
+
+ boolean forEachScope() {
+ return forEachScope;
+ }
+}
diff --git a/src/main/java/com/google/escapevelocity/ConstantExpressionNode.java b/src/main/java/com/google/escapevelocity/ConstantExpressionNode.java
index 50fc9bc..b92e9be 100644
--- a/src/main/java/com/google/escapevelocity/ConstantExpressionNode.java
+++ b/src/main/java/com/google/escapevelocity/ConstantExpressionNode.java
@@ -15,6 +15,8 @@
*/
package com.google.escapevelocity;
+import com.google.common.base.CharMatcher;
+
/**
* A node in the parse tree representing a constant value. Evaluating the node yields the constant
* value. Instances of this class are used both in expressions, like the {@code 23} in
@@ -29,6 +31,8 @@ package com.google.escapevelocity;
* @author emcmanus@google.com (Éamonn McManus)
*/
class ConstantExpressionNode extends ExpressionNode {
+ private static final CharMatcher HORIZONTAL_SPACE =
+ CharMatcher.whitespace().and(CharMatcher.noneOf("\r\n")).precomputed();
private final Object value;
ConstantExpressionNode(String resourceName, int lineNumber, Object value) {
@@ -37,7 +41,17 @@ class ConstantExpressionNode extends ExpressionNode {
}
@Override
- Object evaluate(EvaluationContext context) {
+ public String toString() {
+ return String.valueOf(value);
+ }
+
+ @Override
+ Object evaluate(EvaluationContext context, boolean undefinedIsFalse) {
return value;
}
+
+ @Override
+ boolean isHorizontalWhitespace() {
+ return value instanceof String && HORIZONTAL_SPACE.matchesAllOf((String) value);
+ }
}
diff --git a/src/main/java/com/google/escapevelocity/DirectiveNode.java b/src/main/java/com/google/escapevelocity/DirectiveNode.java
index fd0cd22..c91fff9 100644
--- a/src/main/java/com/google/escapevelocity/DirectiveNode.java
+++ b/src/main/java/com/google/escapevelocity/DirectiveNode.java
@@ -15,7 +15,6 @@
*/
package com.google.escapevelocity;
-import com.google.common.base.Verify;
import com.google.common.collect.ImmutableList;
import java.util.Arrays;
import java.util.Iterator;
@@ -42,18 +41,36 @@ abstract class DirectiveNode extends Node {
*/
static class SetNode extends DirectiveNode {
private final String var;
- private final Node expression;
+ private final ExpressionNode expression;
- SetNode(String var, Node expression) {
+ SetNode(String var, ExpressionNode expression) {
super(expression.resourceName, expression.lineNumber);
this.var = var;
this.expression = expression;
}
- @Override
- Object evaluate(EvaluationContext context) {
+ @Override void render(EvaluationContext context, StringBuilder output) {
context.setVar(var, expression.evaluate(context));
- return "";
+ }
+ }
+
+ /**
+ * A node in the parse tree representing a {@code #define} construct. Evaluating
+ * {@code #define ($x) foo $bar #end} will set {@code $x} to the given sequence of nodes. It does
+ * not produce any text in the output.
+ */
+ static class DefineNode extends DirectiveNode {
+ private final String var;
+ private final Node body;
+
+ DefineNode(String var, Node body) {
+ super(body.resourceName, body.lineNumber);
+ this.var = var;
+ this.body = body;
+ }
+
+ @Override void render(EvaluationContext context, StringBuilder output) {
+ context.setVar(var, body);
}
}
@@ -81,9 +98,9 @@ abstract class DirectiveNode extends Node {
this.falsePart = falseNode;
}
- @Override Object evaluate(EvaluationContext context) {
- Node branch = condition.isDefinedAndTrue(context) ? truePart : falsePart;
- return branch.evaluate(context);
+ @Override void render(EvaluationContext context, StringBuilder output) {
+ Node branch = condition.isTrue(context, /* undefinedIsFalse= */ true) ? truePart : falsePart;
+ branch.render(context, output);
}
}
@@ -93,7 +110,7 @@ abstract class DirectiveNode extends Node {
* turn. Once the loop completes, {@code $x} will go back to whatever value it had before, which
* might be undefined. During loop execution, the variable {@code $foreach} is also defined.
* Velocity defines a number of properties in this variable, but here we only support
- * {@code $foreach.hasNext}.
+ * {@code $foreach.hasNext} and {@code $foreach.index}.
*/
static class ForEachNode extends DirectiveNode {
private final String var;
@@ -108,7 +125,7 @@ abstract class DirectiveNode extends Node {
}
@Override
- Object evaluate(EvaluationContext context) {
+ void render(EvaluationContext context, StringBuilder output) {
Object collectionValue = collection.evaluate(context);
Iterable<?> iterable;
if (collectionValue instanceof Iterable<?>) {
@@ -117,20 +134,24 @@ abstract class DirectiveNode extends Node {
iterable = Arrays.asList((Object[]) collectionValue);
} else if (collectionValue instanceof Map<?, ?>) {
iterable = ((Map<?, ?>) collectionValue).values();
+ } else if (collectionValue == null) {
+ return;
} else {
throw evaluationException("Not iterable: " + collectionValue);
}
Runnable undo = context.setVar(var, null);
- StringBuilder sb = new StringBuilder();
CountingIterator it = new CountingIterator(iterable.iterator());
Runnable undoForEach = context.setVar("foreach", new ForEachVar(it));
- while (it.hasNext()) {
- context.setVar(var, it.next());
- sb.append(body.evaluate(context));
+ try {
+ while (it.hasNext()) {
+ context.setVar(var, it.next());
+ body.render(context, output);
+ }
+ } catch (BreakException e) {
+ // OK: we've broken out of the #foreach
}
undoForEach.run();
undo.run();
- return sb.toString();
}
private static class CountingIterator implements Iterator<Object> {
@@ -161,8 +182,9 @@ abstract class DirectiveNode extends Node {
/**
* This class is the type of the variable {@code $foreach} that is defined within
* {@code #foreach} loops. Its {@link #getHasNext()} method means that we can write
- * {@code #if ($foreach.hasNext)} and likewise for {@link #getIndex()}.
+ * {@code #if ($foreach.hasNext)} and likewise for {@link #getIndex()} etc.
*/
+ @SuppressWarnings("unused") // methods called reflectively
private static class ForEachVar {
private final CountingIterator iterator;
@@ -174,9 +196,58 @@ abstract class DirectiveNode extends Node {
return iterator.hasNext();
}
+ public boolean getFirst() {
+ return iterator.index() == 0;
+ }
+
+ public boolean getLast() {
+ return !iterator.hasNext();
+ }
+
public int getIndex() {
return iterator.index();
}
+
+ public int getCount() {
+ return iterator.index() + 1;
+ }
+
+ // This is consistent with Velocity, if you reference plain $foreach in a template.
+ // (It is "{}" because $foreach is a Scope there, which is a subclass of AbstractMap.)
+ @Override
+ public String toString() {
+ return "{}";
+ }
+ }
+ }
+
+ /**
+ * A node in the parse tree representing a {@code #break} directive. The directive has an optional
+ * scope argument, so you can write {@code #break($foreach)} to break from the nearest
+ * {@code #foreach} loop. That is in fact the only scope we support, so we just have a boolean
+ * that indicates if it is present. Otherwise we break from the nearest {@code #foreach} or
+ * {@code #parse} or from the whole template.
+ */
+ static class BreakNode extends DirectiveNode {
+ private final ExpressionNode scope;
+
+ BreakNode(String resourceName, int lineNumber, ExpressionNode scope) {
+ super(resourceName, lineNumber);
+ this.scope = scope;
+ }
+
+ @Override void render(EvaluationContext context, StringBuilder output) {
+ boolean forEachScope = false;
+ if (scope != null) {
+ Object scopeValue = scope.evaluate(context);
+ if (scopeValue instanceof ForEachNode.ForEachVar) {
+ forEachScope = true;
+ } else {
+ throw evaluationException("Argument to #break is not a supported scope: " + scope);
+ }
+ }
+ String message = "#break " + ParseException.where(resourceName, lineNumber);
+ throw new BreakException(message, forEachScope);
}
}
@@ -192,35 +263,38 @@ abstract class DirectiveNode extends Node {
*/
static class MacroCallNode extends DirectiveNode {
private final String name;
- private final ImmutableList<Node> thunks;
- private Macro macro;
+ private final ImmutableList<ExpressionNode> thunks;
+ private final Node bodyContent;
MacroCallNode(
String resourceName,
int lineNumber,
String name,
- ImmutableList<Node> argumentNodes) {
+ ImmutableList<ExpressionNode> argumentNodes,
+ Node bodyContent) {
super(resourceName, lineNumber);
this.name = name;
this.thunks = argumentNodes;
- }
-
- String name() {
- return name;
- }
-
- int argumentCount() {
- return thunks.size();
- }
-
- void setMacro(Macro macro) {
- this.macro = macro;
+ this.bodyContent = bodyContent;
}
@Override
- Object evaluate(EvaluationContext context) {
- Verify.verifyNotNull(macro, "Macro #%s should have been linked", name);
- return macro.evaluate(context, thunks);
+ void render(EvaluationContext context, StringBuilder output) {
+ Macro macro = context.getMacros().get(name);
+ if (macro == null) {
+ throw evaluationException(
+ "#" + name + " is neither a standard directive nor a macro that has been defined");
+ }
+ if (thunks.size() != macro.parameterCount()) {
+ throw evaluationException(
+ "Wrong number of arguments to #"
+ + name
+ + ": expected "
+ + macro.parameterCount()
+ + ", got "
+ + thunks.size());
+ }
+ macro.render(context, thunks, bodyContent, output);
}
}
}
diff --git a/src/main/java/com/google/escapevelocity/EvaluationContext.java b/src/main/java/com/google/escapevelocity/EvaluationContext.java
index d40b717..b41efea 100644
--- a/src/main/java/com/google/escapevelocity/EvaluationContext.java
+++ b/src/main/java/com/google/escapevelocity/EvaluationContext.java
@@ -24,7 +24,9 @@ import java.util.TreeMap;
* The context of a template evaluation. This consists of the template variables and the template
* macros. The template variables start with the values supplied by the evaluation call, and can
* be changed by {@code #set} directives and during the execution of {@code #foreach} and macro
- * calls. The macros are extracted from the template during parsing and never change thereafter.
+ * calls. The macros start out with the ones that were defined in the root {@link Template} and may
+ * be supplemented by nested templates produced by {@code #parse} directives encountered in the
+ * root template or in nested templates.
*
* @author emcmanus@google.com (Éamonn McManus)
*/
@@ -45,12 +47,22 @@ interface EvaluationContext {
/** See {@link MethodFinder#publicMethodsWithName}. */
ImmutableSet<Method> publicMethodsWithName(Class<?> startClass, String name);
+ /**
+ * Returns a modifiable mapping from macro names to macros. Initially this map starts off with the
+ * macros that were defined in the root template. As {@code #parse} directives are encountered in
+ * the evaluation, they may add further macros to the map.
+ */
+ Map<String, Macro> getMacros();
+
class PlainEvaluationContext implements EvaluationContext {
private final Map<String, Object> vars;
+ private final Map<String, Macro> macros;
private final MethodFinder methodFinder;
- PlainEvaluationContext(Map<String, ?> vars, MethodFinder methodFinder) {
+ PlainEvaluationContext(
+ Map<String, ?> vars, Map<String, Macro> macros, MethodFinder methodFinder) {
this.vars = new TreeMap<>(vars);
+ this.macros = macros;
this.methodFinder = methodFinder;
}
@@ -81,5 +93,10 @@ interface EvaluationContext {
public ImmutableSet<Method> publicMethodsWithName(Class<?> startClass, String name) {
return methodFinder.publicMethodsWithName(startClass, name);
}
+
+ @Override
+ public Map<String, Macro> getMacros() {
+ return macros;
+ }
}
}
diff --git a/src/main/java/com/google/escapevelocity/ExpressionNode.java b/src/main/java/com/google/escapevelocity/ExpressionNode.java
index 281e998..8649b74 100644
--- a/src/main/java/com/google/escapevelocity/ExpressionNode.java
+++ b/src/main/java/com/google/escapevelocity/ExpressionNode.java
@@ -22,6 +22,17 @@ import com.google.escapevelocity.Parser.Operator;
* specifically {@code #set}, {@code #if}, {@code #foreach}, and macro calls. Expressions can
* also appear inside indices in references, like {@code $x[$i]}.
*
+ * <p>Nontrivial expressions are represented by a tree of {@link ExpressionNode} objects. For
+ * example, in {@code #if ($foo.bar < 3)}, the expression {@code $foo.bar < 3} is parsed into
+ * a tree that we might describe as<pre>
+ * {@link BinaryExpressionNode}(
+ * {@link ReferenceNode.MemberReferenceNode}(
+ * {@link ReferenceNode.PlainReferenceNode}("foo"),
+ * "bar"),
+ * {@link Operator#LESS},
+ * {@link ConstantExpressionNode}(3))
+ * </pre>
+ *
* @author emcmanus@google.com (Éamonn McManus)
*/
abstract class ExpressionNode extends Node {
@@ -29,6 +40,60 @@ abstract class ExpressionNode extends Node {
super(resourceName, lineNumber);
}
+ @Override
+ final void render(EvaluationContext context, StringBuilder output) {
+ Object rendered = evaluate(context);
+ if (rendered == null) {
+ if (isSilent()) { // $!foo for example
+ return;
+ }
+ throw evaluationException("Null value for " + this);
+ }
+ if (rendered instanceof Node) {
+ // A macro's $bodyContent, or $x when we earlier did #define ($x) ... #end
+ ((Node) rendered).render(context, output);
+ } else {
+ output.append(rendered);
+ }
+ }
+
+ /**
+ * Returns the source form of this node. This may not be exactly how it appears in the template.
+ * For example both {@code $x} and {@code ${x}} end up being the same kind of node, and its
+ * {@code toString()} is {@code "$x"}.
+ *
+ * <p>This method is used in error messages. It is not invoked in normal template evaluation.
+ */
+ @Override
+ public abstract String toString();
+
+ /**
+ * Returns the result of evaluating this node in the given context. This result may be used as
+ * part of a further operation, for example evaluating {@code 2 + 3} to 5 in order to set
+ * {@code $x} to 5 in {@code #set ($x = 2 + 3)}. Or it may be used directly as part of the
+ * template output, for example evaluating replacing {@code name} by {@code Fred} in
+ * {@code My name is $name.}.
+ *
+ * <p>This has to be an {@code Object} rather than a {@code String} (or rather than appending to
+ * a {@code StringBuilder}) because it can potentially participate in other operations. In the
+ * preceding example, the nodes representing {@code 2} and {@code 3} and the node representing
+ * {@code 2 + 3} all return {@code Integer}. As another example, in {@code #if ($foo.bar < 3)}
+ * the value {@code $foo} is itself an {@link ExpressionNode} which evaluates to the object
+ * referenced by the {@code foo} variable, and {@code $foo.bar} is another {@link ExpressionNode}
+ * that takes the value from {@code foo} and looks for the property {@code bar} in it.
+ *
+ * @param context the context of the evaluation, for example the {@code $variables} that are in
+ * scope
+ * @param undefinedIsFalse whether an undefined plain reference like {@code $foo} is considered
+ * to be false. This is the case when evaluating the condition in an {@code #if}. Everywhere
+ * else, an undefined reference causes an exception.
+ */
+ abstract Object evaluate(EvaluationContext context, boolean undefinedIsFalse);
+
+ final Object evaluate(EvaluationContext context) {
+ return evaluate(context, /* undefinedIsFalse= */ false);
+ }
+
/**
* True if evaluating this expression yields a value that is considered true by Velocity's
* <a href="http://velocity.apache.org/engine/releases/velocity-1.7/user-guide.html#Conditionals">
@@ -38,10 +103,10 @@ abstract class ExpressionNode extends Node {
* <p>Note that the text at the similar link
* <a href="http://velocity.apache.org/engine/devel/user-guide.html#Conditionals">here</a>
* states that empty collections and empty strings are also considered false, but that is not
- * true.
+ * what Velocity actually implements.
*/
- boolean isTrue(EvaluationContext context) {
- Object value = evaluate(context);
+ boolean isTrue(EvaluationContext context, boolean undefinedIsFalse) {
+ Object value = evaluate(context, undefinedIsFalse);
if (value instanceof Boolean) {
return (Boolean) value;
} else {
@@ -50,26 +115,26 @@ abstract class ExpressionNode extends Node {
}
/**
- * True if this is a defined value and it evaluates to true. This is the same as {@link #isTrue}
- * except that it is allowed for this to be undefined variable, in which it evaluates to false.
- * The method is overridden for plain references so that undefined is the same as false.
- * The reason is to support Velocity's idiom {@code #if ($var)}, where it is not an error
- * if {@code $var} is undefined.
+ * True if a null value for this expression is silently translated to an empty string when
+ * substituted into template text. Otherwise it results in an exception.
*/
- boolean isDefinedAndTrue(EvaluationContext context) {
- return isTrue(context);
+ boolean isSilent() {
+ return false;
}
/**
- * The integer result of evaluating this expression.
+ * The integer result of evaluating this expression, or null if the expression evaluates to null.
*
* @throws EvaluationException if evaluating the expression produces an exception, or if it
- * yields a value that is not an integer.
+ * yields a value that is neither an integer nor null.
*/
- int intValue(EvaluationContext context) {
+ Integer intValue(EvaluationContext context) {
Object value = evaluate(context);
+ if (value == null) {
+ return null;
+ }
if (!(value instanceof Integer)) {
- throw evaluationException("Arithemtic is only available on integers, not " + show(value));
+ throw evaluationException("Arithmetic is only available on integers, not " + show(value));
}
return (Integer) value;
}
@@ -102,20 +167,41 @@ abstract class ExpressionNode extends Node {
this.rhs = rhs;
}
- @Override Object evaluate(EvaluationContext context) {
+ @Override public String toString() {
+ return operandString(lhs) + " " + op + " " + operandString(rhs);
+ }
+
+ // Restore the parentheses in, for example, (2 + 3) * 4.
+ private String operandString(ExpressionNode operand) {
+ String s = String.valueOf(operand);
+ if (operand instanceof BinaryExpressionNode) {
+ BinaryExpressionNode binaryOperand = (BinaryExpressionNode) operand;
+ if (binaryOperand.op.precedence < op.precedence) {
+ return "(" + s + ")";
+ }
+ }
+ return s;
+ }
+
+ @Override Object evaluate(EvaluationContext context, boolean undefinedIsFalse) {
switch (op) {
case OR:
- return lhs.isTrue(context) || rhs.isTrue(context);
+ return lhs.isTrue(context, undefinedIsFalse) || rhs.isTrue(context, undefinedIsFalse);
case AND:
- return lhs.isTrue(context) && rhs.isTrue(context);
+ return lhs.isTrue(context, undefinedIsFalse) && rhs.isTrue(context, undefinedIsFalse);
case EQUAL:
return equal(context);
case NOT_EQUAL:
return !equal(context);
+ case PLUS:
+ return plus(context);
default: // fall out
}
- int lhsInt = lhs.intValue(context);
- int rhsInt = rhs.intValue(context);
+ Integer lhsInt = lhs.intValue(context);
+ Integer rhsInt = rhs.intValue(context);
+ if (lhsInt == null || rhsInt == null) {
+ return nullOperand(lhsInt == null);
+ }
switch (op) {
case LESS:
return lhsInt < rhsInt;
@@ -125,21 +211,29 @@ abstract class ExpressionNode extends Node {
return lhsInt > rhsInt;
case GREATER_OR_EQUAL:
return lhsInt >= rhsInt;
- case PLUS:
- return lhsInt + rhsInt;
case MINUS:
return lhsInt - rhsInt;
case TIMES:
return lhsInt * rhsInt;
case DIVIDE:
- return lhsInt / rhsInt;
+ return (rhsInt == 0) ? null : lhsInt / rhsInt;
case REMAINDER:
- return lhsInt % rhsInt;
+ return (rhsInt == 0) ? null : lhsInt % rhsInt;
default:
throw new AssertionError(op);
}
}
+ // Mimic Velocity's null-handling.
+ private Void nullOperand(boolean leftIsNull) {
+ if (op.isInequality()) {
+ // If both are null we'll only complain about the left one.
+ String operand = leftIsNull ? "Left operand " + lhs : "Right operand " + rhs;
+ throw evaluationException(operand + " of " + op + " must not be null");
+ }
+ return null;
+ }
+
/**
* Returns true if {@code lhs} and {@code rhs} are equal according to Velocity.
*
@@ -168,6 +262,35 @@ abstract class ExpressionNode extends Node {
// Funky equals behaviour specified by Velocity.
return lhsValue.toString().equals(rhsValue.toString());
}
+
+ private Object plus(EvaluationContext context) {
+ Object lhsValue = lhs.evaluate(context);
+ Object rhsValue = rhs.evaluate(context);
+ if (lhsValue instanceof String || rhsValue instanceof String) {
+ // Velocity's treatment of null is all over the map. In a string concatenation, a null
+ // reference is replaced by the the source text of the reference, for example "$foo". The
+ // toString() that we have for the various ExpressionNode subtypes reproduces this at least
+ // in our test cases.
+ if (lhsValue == null) {
+ lhsValue = lhs.toString();
+ }
+ if (rhsValue == null) {
+ rhsValue = rhs.toString();
+ }
+ return new StringBuilder().append(lhsValue).append(rhsValue).toString();
+ }
+ if (lhsValue == null || rhsValue == null) {
+ return null;
+ }
+ if (!(lhsValue instanceof Integer) || !(rhsValue instanceof Integer)) {
+ throw evaluationException(
+ "Operands of + must both be integers, or at least one must be a string: "
+ + show(lhsValue)
+ + " + "
+ + show(rhsValue));
+ }
+ return (Integer) lhsValue + (Integer) rhsValue;
+ }
}
/**
@@ -181,8 +304,15 @@ abstract class ExpressionNode extends Node {
this.expr = expr;
}
- @Override Object evaluate(EvaluationContext context) {
- return !expr.isTrue(context);
+ @Override public String toString() {
+ if (expr instanceof BinaryExpressionNode) {
+ return "!(" + expr + ")";
+ }
+ return "!" + expr;
+ }
+
+ @Override Object evaluate(EvaluationContext context, boolean undefinedIsFalse) {
+ return !expr.isTrue(context, undefinedIsFalse);
}
}
}
diff --git a/src/main/java/com/google/escapevelocity/Macro.java b/src/main/java/com/google/escapevelocity/Macro.java
index afa7bf0..108689e 100644
--- a/src/main/java/com/google/escapevelocity/Macro.java
+++ b/src/main/java/com/google/escapevelocity/Macro.java
@@ -30,43 +30,54 @@ import java.util.Map;
* means that we need to set each parameter variable to the node in the parse tree that corresponds
* to it, and arrange for that node to be evaluated when the variable is actually referenced.
*
+ * <p>There are two ways to invoke a macro. {@code #m('foo', 'bar')} sets $x and $y. {@code
+ * #@m('foo', 'bar') ... #end} sets $x and $y, and also sets $bodyContent to the template text
+ * {@code ...}.
+ *
* @author emcmanus@google.com (Éamonn McManus)
*/
class Macro {
private final int definitionLineNumber;
private final String name;
private final ImmutableList<String> parameterNames;
- private final Node body;
+ private final Node macroBody;
- Macro(int definitionLineNumber, String name, List<String> parameterNames, Node body) {
+ Macro(int definitionLineNumber, String name, List<String> parameterNames, Node macroBody) {
this.definitionLineNumber = definitionLineNumber;
this.name = name;
this.parameterNames = ImmutableList.copyOf(parameterNames);
- this.body = body;
- }
-
- String name() {
- return name;
+ this.macroBody = macroBody;
}
int parameterCount() {
return parameterNames.size();
}
- Object evaluate(EvaluationContext context, List<Node> thunks) {
+ /**
+ * Renders a call to this macro with the arguments in {@code thunks} and with a possibly-null
+ * {@code bodyContent}. The {@code bodyContent} is non-null if the macro call looks like
+ * {@code #@foo(...) ... #end}; the {@code #@} indicates that the text up to the matching
+ * {@code #end} should be made available as the variable {@code $bodyContent} inside the macro.
+ */
+ void render(
+ EvaluationContext context,
+ List<ExpressionNode> thunks,
+ Node bodyContent,
+ StringBuilder output) {
try {
- Verify.verify(thunks.size() == parameterNames.size(), "Argument mistmatch for %s", name);
- Map<String, Node> parameterThunks = new LinkedHashMap<>();
+ Verify.verify(thunks.size() == parameterNames.size(), "Argument mismatch for %s", name);
+ Map<String, ExpressionNode> parameterThunks = new LinkedHashMap<>();
for (int i = 0; i < parameterNames.size(); i++) {
parameterThunks.put(parameterNames.get(i), thunks.get(i));
}
- EvaluationContext newContext = new MacroEvaluationContext(parameterThunks, context);
- return body.evaluate(newContext);
+ EvaluationContext newContext =
+ new MacroEvaluationContext(parameterThunks, context, bodyContent);
+ macroBody.render(newContext, output);
} catch (EvaluationException e) {
EvaluationException newException = new EvaluationException(
"In macro #" + name + " defined on line " + definitionLineNumber + ": " + e.getMessage());
newException.setStackTrace(e.getStackTrace());
- throw e;
+ throw newException;
}
}
@@ -87,18 +98,25 @@ class Macro {
* but it has the same responsibility.
*/
static class MacroEvaluationContext implements EvaluationContext {
- private final Map<String, Node> parameterThunks;
+ private final Map<String, ExpressionNode> parameterThunks;
private final EvaluationContext originalEvaluationContext;
+ private final Node bodyContent;
MacroEvaluationContext(
- Map<String, Node> parameterThunks, EvaluationContext originalEvaluationContext) {
+ Map<String, ExpressionNode> parameterThunks,
+ EvaluationContext originalEvaluationContext,
+ Node bodyContent) {
this.parameterThunks = parameterThunks;
this.originalEvaluationContext = originalEvaluationContext;
+ this.bodyContent = bodyContent;
}
@Override
public Object getVar(String var) {
- Node thunk = parameterThunks.get(var);
+ if (bodyContent != null && var.equals("bodyContent")) {
+ return bodyContent;
+ }
+ ExpressionNode thunk = parameterThunks.get(var);
if (thunk == null) {
return originalEvaluationContext.getVar(var);
} else {
@@ -113,14 +131,16 @@ class Macro {
@Override
public boolean varIsDefined(String var) {
- return parameterThunks.containsKey(var) || originalEvaluationContext.varIsDefined(var);
+ return parameterThunks.containsKey(var)
+ || (bodyContent != null && var.equals("bodyContent"))
+ || originalEvaluationContext.varIsDefined(var);
}
@Override
public Runnable setVar(final String var, Object value) {
// Copy the behaviour that #set will shadow a macro parameter, even though the Velocity peeps
// seem to agree that that is not good.
- final Node thunk = parameterThunks.get(var);
+ final ExpressionNode thunk = parameterThunks.get(var);
if (thunk == null) {
return originalEvaluationContext.setVar(var, value);
} else {
@@ -137,5 +157,10 @@ class Macro {
public ImmutableSet<Method> publicMethodsWithName(Class<?> startClass, String name) {
return originalEvaluationContext.publicMethodsWithName(startClass, name);
}
+
+ @Override
+ public Map<String, Macro> getMacros() {
+ return originalEvaluationContext.getMacros();
+ }
}
}
diff --git a/src/main/java/com/google/escapevelocity/Node.java b/src/main/java/com/google/escapevelocity/Node.java
index a017afa..117910f 100644
--- a/src/main/java/com/google/escapevelocity/Node.java
+++ b/src/main/java/com/google/escapevelocity/Node.java
@@ -32,13 +32,16 @@ abstract class Node {
}
/**
- * Returns the result of evaluating this node in the given context. This result may be used as
- * part of a further operation, for example evaluating {@code 2 + 3} to 5 in order to set
- * {@code $x} to 5 in {@code #set ($x = 2 + 3)}. Or it may be used directly as part of the
- * template output, for example evaluating replacing {@code name} by {@code Fred} in
- * {@code My name is $name.}.
+ * Adds this node's contribution to the {@code output}. Depending on the node type, this might be
+ * plain text from the template, or one branch of an {@code #if}, or the value of a
+ * {@code $reference}, etc.
*/
- abstract Object evaluate(EvaluationContext context);
+ abstract void render(EvaluationContext context, StringBuilder output);
+
+ /** True if this node is just a span of horizontal whitespace in the text. */
+ boolean isHorizontalWhitespace() {
+ return false;
+ }
private String where() {
String where = "In expression on line " + lineNumber;
@@ -64,6 +67,7 @@ abstract class Node {
return new Cons(resourceName, lineNumber, ImmutableList.<Node>of());
}
+
/**
* Create a new parse tree node that is the concatenation of the given ones. Evaluating the
* new node produces the same string as evaluating each of the given nodes and concatenating the
@@ -81,12 +85,11 @@ abstract class Node {
this.nodes = nodes;
}
- @Override Object evaluate(EvaluationContext context) {
- StringBuilder sb = new StringBuilder();
+ @Override
+ void render(EvaluationContext context, StringBuilder output) {
for (Node node : nodes) {
- sb.append(node.evaluate(context));
+ node.render(context, output);
}
- return sb.toString();
}
}
}
diff --git a/src/main/java/com/google/escapevelocity/ParseException.java b/src/main/java/com/google/escapevelocity/ParseException.java
index 7105f97..f328355 100644
--- a/src/main/java/com/google/escapevelocity/ParseException.java
+++ b/src/main/java/com/google/escapevelocity/ParseException.java
@@ -31,7 +31,7 @@ public class ParseException extends RuntimeException {
super(message + ", " + where(resourceName, lineNumber) + ", at text starting: " + context);
}
- private static String where(String resourceName, int lineNumber) {
+ static String where(String resourceName, int lineNumber) {
if (resourceName == null) {
return "on line " + lineNumber;
} else {
diff --git a/src/main/java/com/google/escapevelocity/ParseNode.java b/src/main/java/com/google/escapevelocity/ParseNode.java
new file mode 100644
index 0000000..e3809cb
--- /dev/null
+++ b/src/main/java/com/google/escapevelocity/ParseNode.java
@@ -0,0 +1,82 @@
+/*
+ * Copyright (C) 2023 Google, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.google.escapevelocity;
+
+import com.google.escapevelocity.Template.ResourceOpener;
+import java.io.IOException;
+import java.util.Map;
+
+/**
+ * Representation of a {@code #parse} directive in the parse tree.
+ *
+ * <p>A {@code #parse (<primary>)} directive does nothing until the template is evaluated. Then the
+ * {@code <primary>} should evaluate to a string, which will be used to derive a nested {@link
+ * Template}. If the string was seen before (in this evaluation, or an earlier evaluation) then the
+ * {@code Template} will be retrieved from the {@code parseCache}. Otherwise we will get a {@link
+ * Reader} from the {@link ResourceOpener} and parse its contents to produce a new {@code Template},
+ * which we will record in the {@code parseCache}. Either way, we will execute the nested {@code
+ * Template}, which means adding its macros to the {@link EvaluationContext} and evaluating it to
+ * add text to the final output.
+ */
+class ParseNode extends Node {
+ private final ExpressionNode nestedResourceNameExpression;
+ private final ResourceOpener resourceOpener;
+ private final Map<String, Template> parseCache;
+
+ ParseNode(
+ String resourceName,
+ int startLine,
+ ExpressionNode nestedResourceNameExpression,
+ ResourceOpener resourceOpener,
+ Map<String, Template> parseCache) {
+ super(resourceName, startLine);
+ this.nestedResourceNameExpression = nestedResourceNameExpression;
+ this.resourceOpener = resourceOpener;
+ this.parseCache = parseCache;
+ }
+
+ @Override
+ void render(EvaluationContext context, StringBuilder output) {
+ Object resourceNameObject = nestedResourceNameExpression.evaluate(context);
+ if (!(resourceNameObject instanceof String)) {
+ String what = resourceNameObject == null ? "null" : resourceNameObject.getClass().getName();
+ throw evaluationException("Argument to #parse must be a string, not " + what);
+ }
+ String resourceName = (String) resourceNameObject;
+ Template template = parseCache.get(resourceName);
+ if (template == null) {
+ try {
+ template = Template.parseFrom(resourceName, resourceOpener, parseCache);
+ parseCache.put(resourceName, template);
+ } catch (ParseException | IOException e) {
+ throw evaluationException(e);
+ }
+ }
+
+ // Lift macros from the nested template into the evaluation context so we can evaluate the
+ // nested template with the pre-processed macros available. The macros will also be available in
+ // the calling template after the #parse.
+ template.getMacros().forEach((name, macro) -> context.getMacros().putIfAbsent(name, macro));
+ try {
+ template.render(context, output);
+ } catch (BreakException e) {
+ if (e.forEachScope()) { // this isn't for us
+ throw e;
+ }
+ // OK: the #break has broken out of this #parse
+ }
+ }
+}
diff --git a/src/main/java/com/google/escapevelocity/Parser.java b/src/main/java/com/google/escapevelocity/Parser.java
index 4416c48..1a5f4f0 100644
--- a/src/main/java/com/google/escapevelocity/Parser.java
+++ b/src/main/java/com/google/escapevelocity/Parser.java
@@ -16,12 +16,22 @@
package com.google.escapevelocity;
import com.google.common.base.CharMatcher;
+import com.google.common.base.Joiner;
import com.google.common.base.Verify;
+import com.google.common.collect.ContiguousSet;
+import com.google.common.collect.ForwardingSortedSet;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableListMultimap;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.ImmutableSortedSet;
import com.google.common.collect.Iterables;
import com.google.common.primitives.Chars;
import com.google.common.primitives.Ints;
+import com.google.escapevelocity.DirectiveNode.BreakNode;
+import com.google.escapevelocity.DirectiveNode.DefineNode;
+import com.google.escapevelocity.DirectiveNode.ForEachNode;
+import com.google.escapevelocity.DirectiveNode.IfNode;
import com.google.escapevelocity.DirectiveNode.SetNode;
import com.google.escapevelocity.ExpressionNode.BinaryExpressionNode;
import com.google.escapevelocity.ExpressionNode.NotExpressionNode;
@@ -29,18 +39,20 @@ import com.google.escapevelocity.ReferenceNode.IndexReferenceNode;
import com.google.escapevelocity.ReferenceNode.MemberReferenceNode;
import com.google.escapevelocity.ReferenceNode.MethodReferenceNode;
import com.google.escapevelocity.ReferenceNode.PlainReferenceNode;
-import com.google.escapevelocity.TokenNode.CommentTokenNode;
-import com.google.escapevelocity.TokenNode.ElseIfTokenNode;
-import com.google.escapevelocity.TokenNode.ElseTokenNode;
-import com.google.escapevelocity.TokenNode.EndTokenNode;
-import com.google.escapevelocity.TokenNode.EofNode;
-import com.google.escapevelocity.TokenNode.ForEachTokenNode;
-import com.google.escapevelocity.TokenNode.IfTokenNode;
-import com.google.escapevelocity.TokenNode.MacroDefinitionTokenNode;
-import com.google.escapevelocity.TokenNode.NestedTokenNode;
+import com.google.escapevelocity.StopNode.ElseIfNode;
+import com.google.escapevelocity.StopNode.ElseNode;
+import com.google.escapevelocity.StopNode.EndNode;
+import com.google.escapevelocity.StopNode.EofNode;
import java.io.IOException;
import java.io.LineNumberReader;
import java.io.Reader;
+import java.io.StringReader;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.TreeMap;
+import java.util.function.Supplier;
/**
* A parser that reads input from the given {@link Reader} and parses it to produce a
@@ -51,10 +63,32 @@ import java.io.Reader;
class Parser {
private static final int EOF = -1;
+ private static final ImmutableSet<Class<? extends StopNode>> EOF_CLASS =
+ ImmutableSet.of(EofNode.class);
+ private static final ImmutableSet<Class<? extends StopNode>> END_CLASS =
+ ImmutableSet.of(EndNode.class);
+ private static final ImmutableSet<Class<? extends StopNode>> ELSE_ELSEIF_END_CLASSES =
+ ImmutableSet.of(ElseNode.class, ElseIfNode.class, EndNode.class);
+
private final LineNumberReader reader;
private final String resourceName;
private final Template.ResourceOpener resourceOpener;
+ /**
+ * Map from resource name to already-parsed template. This map is shared between all of the nested
+ * {@link Parser} instances that result from {@code #parse} directives, so we will only ever read
+ * and parse any given resource name once.
+ */
+ private final Map<String, Template> parseCache;
+
+ /**
+ * Macros that have been defined during this parse. This means macros defined in a given {@code
+ * foo.vm} file, without regard to whatever macros might be defined in another {@code bar.vm}
+ * file. If the same name is defined more than once in {@code foo.vm}, only the first definition
+ * has any effect.
+ */
+ private final Map<String, Macro> macros = new TreeMap<>();
+
/**
* The invariant of this parser is that {@code c} is always the next character of interest.
* This means that we almost never have to "unget" a character by reading too far. For example,
@@ -71,62 +105,29 @@ class Parser {
*/
private int pushback = -1;
- Parser(Reader reader, String resourceName, Template.ResourceOpener resourceOpener)
+ Parser(
+ Reader reader,
+ String resourceName,
+ Template.ResourceOpener resourceOpener,
+ Map<String, Template> parseCache)
throws IOException {
this.reader = new LineNumberReader(reader);
this.reader.setLineNumber(1);
next();
this.resourceName = resourceName;
this.resourceOpener = resourceOpener;
+ this.parseCache = parseCache;
}
/**
- * Parse the input completely to produce a {@link Template}.
- *
- * <p>Parsing happens in two phases. First, we parse a sequence of "tokens", where tokens include
- * entire references such as <pre>
- * ${x.foo()[23]}
- * </pre>or entire directives such as<pre>
- * #set ($x = $y + $z)
- * </pre>But tokens do not span complex constructs. For example,<pre>
- * #if ($x == $y) something #end
- * </pre>is three tokens:<pre>
- * #if ($x == $y)
- * (literal text " something ")
- * #end
- * </pre>
- *
- * <p>The second phase then takes the sequence of tokens and constructs a parse tree out of it.
- * Some nodes in the parse tree will be unchanged from the token sequence, such as the <pre>
- * ${x.foo()[23]}
- * #set ($x = $y + $z)
- * </pre> examples above. But a construct such as the {@code #if ... #end} mentioned above will
- * become a single IfNode in the parse tree in the second phase.
- *
- * <p>The main reason for this approach is that Velocity has two kinds of lexical contexts. At the
- * top level, there can be arbitrary literal text; references like <code>${x.foo()}</code>; and
- * directives like {@code #if} or {@code #set}. Inside the parentheses of a directive, however,
- * neither arbitrary text nor directives can appear, but expressions can, so we need to tokenize
- * the inside of <pre>
- * #if ($x == $a + $b)
- * </pre> as the five tokens "$x", "==", "$a", "+", "$b". Rather than having a classical
- * parser/lexer combination, where the lexer would need to switch between these two modes, we
- * replace the lexer with an ad-hoc parser that is the first phase described above, and we
- * define a simple parser over the resultant tokens that is the second phase.
+ * Parse the input completely to produce a {@link Template}. We use a fairly standard
+ * recursive-descent parser with ad-hoc lexing and a few hacks needed to reproduce quirks of
+ * Velocity's behaviour.
*/
Template parse() throws IOException {
- ImmutableList<Node> tokens = parseTokens();
- return new Reparser(tokens).reparse();
- }
-
- private ImmutableList<Node> parseTokens() throws IOException {
- ImmutableList.Builder<Node> tokens = ImmutableList.builder();
- Node token;
- do {
- token = parseNode();
- tokens.add(token);
- } while (!(token instanceof EofNode));
- return tokens.build();
+ ParseResult parseResult = parseToStop(EOF_CLASS, () -> "outside any construct");
+ Node root = Node.cons(resourceName, lineNumber(), parseResult.nodes);
+ return new Template(root, ImmutableMap.copyOf(macros));
}
private int lineNumber() {
@@ -193,14 +194,64 @@ class Parser {
}
}
+ private static class ParseResult {
+ final ImmutableList<Node> nodes;
+ final StopNode stop;
+
+ ParseResult(ImmutableList<Node> nodes, StopNode stop) {
+ this.nodes = nodes;
+ this.stop = stop;
+ }
+ }
+
/**
- * Parses a single node from the reader, as part of the first parsing phase.
- * <pre>{@code
- * <template> -> <empty> |
- * <directive> <template> |
- * <non-directive> <template>
- * }</pre>
+ * Parse until reaching a {@code StopNode}. The {@code StopNode} must have one of the classes in
+ * {@code stopClasses}. This method is called recursively to parse nested constructs. At the
+ * top level, we expect the parse to end when it reaches {@code EofNode}. In a {@code #foreach},
+ * for example, we expect the parse to end when it reaches the matching {@code #end}. In an
+ * {@code #if}, the parse can end with {@code #end}, {@code #else}, or {@code #elseif}. And then
+ * after {@code #else} or {@code #elseif} we will call this method again to parse the next part.
+ *
+ * @return the nodes that were parsed, plus the {@code StopNode} that caused parsing to stop.
+ */
+ private ParseResult parseToStop(
+ ImmutableSet<Class<? extends StopNode>> stopClasses, Supplier<String> contextDescription)
+ throws IOException {
+ List<Node> nodes = new ArrayList<>();
+ Node node;
+ while (true) {
+ node = parseNode();
+ if (node instanceof StopNode) {
+ break;
+ }
+ if (node instanceof SetNode && SetSpacing.shouldRemoveLastNodeBeforeSet(nodes)) {
+ nodes.set(nodes.size() - 1, node);
+ } else {
+ nodes.add(node);
+ }
+ }
+ StopNode stop = (StopNode) node;
+ if (!stopClasses.contains(stop.getClass())) {
+ throw parseException("Found " + stop.name() + " " + contextDescription.get());
+ }
+ return new ParseResult(ImmutableList.copyOf(nodes), stop);
+ }
+
+ /**
+ * Skip the current character if it is a newline, then parse until reaching a {@code StopNode}.
+ * This is used after directives like {@code #if}, where a newline is ignored after the final
+ * {@code )} in {@code #if (condition)}.
*/
+ private ParseResult skipNewlineAndParseToStop(
+ ImmutableSet<Class<? extends StopNode>> stopClasses, Supplier<String> contextDescription)
+ throws IOException {
+ if (c == '\n') {
+ next();
+ }
+ return parseToStop(stopClasses, contextDescription);
+ }
+
+ /** Parses a single node from the reader. */
private Node parseNode() throws IOException {
if (c == '#') {
next();
@@ -213,6 +264,8 @@ class Parser {
return parseHashSquare();
case '{':
return parseDirective();
+ case '@':
+ return parseMacroCallWithBody();
default:
if (isAsciiLetter(c)) {
return parseDirective();
@@ -263,20 +316,13 @@ class Parser {
}
/**
- * Parses a single non-directive node from the reader.
- * <pre>{@code
- * <non-directive> -> <reference> |
- * <text containing neither $ nor #>
- * }</pre>
+ * Parses a single non-directive node from the reader. This is either a reference, like
+ * {@code $foo} or {@code $bar.baz} or {@code $foo.bar[$baz].buh()}; or it is text containing
+ * neither references (no {@code $}) nor directives (no {@code #}).
*/
private Node parseNonDirective() throws IOException {
if (c == '$') {
- next();
- if (isAsciiLetter(c) || c == '{') {
- return parseReference();
- } else {
- return parsePlainText('$');
- }
+ return parseDollar();
} else {
int firstChar = c;
next();
@@ -284,21 +330,27 @@ class Parser {
}
}
+ private Node parseDollar() throws IOException {
+ assert c == '$';
+ next();
+ boolean silent = c == '!';
+ if (silent) {
+ next();
+ }
+ if (isAsciiLetter(c) || c == '{') {
+ return parseReference(silent);
+ } else if (silent) {
+ return parsePlainText("$!");
+ } else {
+ return parsePlainText('$');
+ }
+ }
+
/**
* Parses a single directive token from the reader. Directives can be spelled with or without
- * braces, for example {@code #if} or {@code #{if}}. We omit the brace spelling in the productions
- * here: <pre>{@code
- * <directive> -> <if-token> |
- * <else-token> |
- * <elseif-token> |
- * <end-token> |
- * <foreach-token> |
- * <set-token> |
- * <parse-token> |
- * <macro-token> |
- * <macro-call> |
- * <comment>
- * }</pre>
+ * braces, for example {@code #if} or {@code #{if}}. In the case of {@code #end}, {@code #else},
+ * and {@code #elseif}, we return a {@link StopNode} representing just the token itself. In other
+ * cases we also parse the complete directive, for example a complete {@code #foreach...#end}.
*/
private Node parseDirective() throws IOException {
String directive;
@@ -312,31 +364,39 @@ class Parser {
Node node;
switch (directive) {
case "end":
- node = new EndTokenNode(resourceName, lineNumber());
+ node = new EndNode(resourceName, lineNumber());
break;
case "if":
+ return parseIfOrElseIf("#if");
case "elseif":
- node = parseIfOrElseIf(directive);
+ node = new ElseIfNode(resourceName, lineNumber());
break;
case "else":
- node = new ElseTokenNode(resourceName, lineNumber());
+ node = new ElseNode(resourceName, lineNumber());
break;
case "foreach":
- node = parseForEach();
- break;
+ return parseForEach();
+ case "break":
+ return parseBreak();
case "set":
node = parseSet();
break;
+ case "define":
+ node = parseDefine();
+ break;
case "parse":
node = parseParse();
break;
case "macro":
- node = parseMacroDefinition();
- break;
+ return parseMacroDefinition();
+ case "evaluate":
+ return parseEvaluate();
default:
- node = parsePossibleMacroCall(directive);
+ node = parseMacroCall("#", directive);
}
- // Velocity skips a newline after any directive.
+ // Velocity skips a newline after any directive. In the case of #if etc, we'll have done this
+ // when we stopped scanning the body at #end, so in those cases we return directly rather than
+ // breaking into the code here.
// TODO(emcmanus): in fact it also skips space before the newline, which should be implemented.
if (c == '\n') {
next();
@@ -345,30 +405,57 @@ class Parser {
}
/**
- * Parses the condition following {@code #if} or {@code #elseif}.
+ * Parses an {@code #if} construct, or an {@code #elseif} within one.
+ *
* <pre>{@code
- * <if-token> -> #if ( <condition> )
- * <elseif-token> -> #elseif ( <condition> )
+ * #if ( <condition> ) <true-text> #end
+ * #if ( <condition> ) <true-text> #else <false-text> #end
+ * #if ( <condition1> ) <text1> #elseif ( <condition2> ) <text2> #else <text3> #end
* }</pre>
- *
- * @param directive either {@code "if"} or {@code "elseif"}.
*/
private Node parseIfOrElseIf(String directive) throws IOException {
+ int startLine = lineNumber();
expect('(');
ExpressionNode condition = parseExpression();
expect(')');
- return directive.equals("if") ? new IfTokenNode(condition) : new ElseIfTokenNode(condition);
+ ParseResult parsedTruePart =
+ skipNewlineAndParseToStop(
+ ELSE_ELSEIF_END_CLASSES,
+ () -> "parsing " + directive + " starting on line " + startLine);
+ Node truePart = Node.cons(resourceName, startLine, parsedTruePart.nodes);
+ Node falsePart;
+ if (parsedTruePart.stop instanceof EndNode) {
+ falsePart = Node.emptyNode(resourceName, lineNumber());
+ } else if (parsedTruePart.stop instanceof ElseIfNode) {
+ falsePart = parseIfOrElseIf("#elseif");
+ } else {
+ int elseLine = lineNumber();
+ ParseResult parsedFalsePart =
+ parseToStop(END_CLASS, () -> "parsing #else starting on line " + elseLine);
+ falsePart = Node.cons(resourceName, elseLine, parsedFalsePart.nodes);
+ }
+ return new IfNode(resourceName, startLine, condition, truePart, falsePart);
}
/**
- * Parses a {@code #foreach} token from the reader. <pre>{@code
- * <foreach-token> -> #foreach ( $<id> in <expression> )
+ * Parses a {@code #foreach} token from the reader.
+ *
+ * <pre>{@code
+ * #foreach ( $<id> in <expression> ) <body> #end
* }</pre>
*/
private Node parseForEach() throws IOException {
+ int startLine = lineNumber();
expect('(');
- expect('$');
- String var = parseId("For-each variable");
+ skipSpace();
+ if (c != '$') {
+ throw parseException("Expected variable beginning with '$' for #foreach");
+ }
+ Node varNode = parseDollar();
+ if (!(varNode instanceof PlainReferenceNode)) {
+ throw parseException("Expected simple variable for #foreach");
+ }
+ String var = ((PlainReferenceNode) varNode).id;
skipSpace();
boolean bad = false;
if (c != 'i') {
@@ -385,12 +472,42 @@ class Parser {
next();
ExpressionNode collection = parseExpression();
expect(')');
- return new ForEachTokenNode(var, collection);
+ ParseResult parsedBody =
+ skipNewlineAndParseToStop(
+ END_CLASS, () -> "parsing #foreach starting on line " + startLine);
+ Node body = Node.cons(resourceName, startLine, parsedBody.nodes);
+ return new ForEachNode(resourceName, startLine, var, collection, body);
+ }
+
+ /**
+ * Parses a {@code #break} token from the reader.
+ *
+ * <p>There is an optional scope, so you can write {@code #break ($foreach)},
+ * {@code #break ($foreach.parent)}, {@code #break ($parse)}, and so on. We only support
+ * {@code $foreach}. If there is no scope, we will break from the nearest {@code #foreach} or
+ * {@code #parse}, or, if there is none, from the whole template.
+ */
+ private Node parseBreak() throws IOException {
+ // Unlike every other directive, #break has an *optional* parenthesized parameter. But even if
+ // we *don't* see a `(` after skipping spaces, we can safely discard the spaces. It's a #break,
+ // so any plain text after it will never be rendered anyway. (We could even discard any
+ // non-space plain text, but it's probably not worth bothering.) For the same reason, we don't
+ // need to skip a \n that might occur after the #break.
+ skipSpace();
+ ExpressionNode scope = null;
+ if (c == '(') {
+ next();
+ scope = parsePrimary();
+ expect(')');
+ }
+ return new BreakNode(resourceName, lineNumber(), scope);
}
/**
- * Parses a {@code #set} token from the reader. <pre>{@code
- * <set-token> -> #set ( $<id> = <expression>)
+ * Parses a {@code #set} token from the reader.
+ *
+ * <pre>{@code
+ * #set ( $<id> = <expression> )
* }</pre>
*/
private Node parseSet() throws IOException {
@@ -404,44 +521,57 @@ class Parser {
}
/**
- * Parses a {@code #parse} token from the reader. <pre>{@code
- * <parse-token> -> #parse ( <string-literal> )
+ * Parses a {@code #define} directive from the reader.
+ *
+ * <pre>{@code
+ * #define ( $<id> ) <balanced-tokens> #end
* }</pre>
+ */
+ private Node parseDefine() throws IOException {
+ int startLine = lineNumber();
+ expect('(');
+ expect('$');
+ String var = parseId("#define variable");
+ expect(')');
+ ParseResult parseResult =
+ skipNewlineAndParseToStop(END_CLASS, () -> "parsing #define starting on line " + startLine);
+ return new DefineNode(var, Node.cons(resourceName, startLine, parseResult.nodes));
+ }
+
+ /**
+ * Parses a {@code #parse} token from the reader.
*
- * <p>The way this works is inconsistent with Velocity. In Velocity, the {@code #parse} directive
- * is evaluated when it is encountered during template evaluation. That means that the argument
- * can be a variable, and it also means that you can use {@code #if} to choose whether or not
- * to do the {@code #parse}. Neither of those is true in EscapeVelocity. The contents of the
- * {@code #parse} are integrated into the containing template pretty much as if they had been
- * written inline. That also means that EscapeVelocity allows forward references to macros
- * inside {@code #parse} directives, which Velocity does not.
+ * <pre>{@code
+ * #parse ( <primary> )
+ * }</pre>
+ *
+ * <p>When we see a {@code #parse} directive while parsing a template, all we do is record it as a
+ * {@link ParseNode} in the {@link Template} we produce. We only actually open and parse the
+ * resource named in the {@code #parse} when the template is later <i>evaluated</i>. The {@code
+ * parseCache} means that we will only do this once, at least if the argument to the {@code
+ * #parse} is always the same string.
*/
private Node parseParse() throws IOException {
+ int startLine = lineNumber();
expect('(');
+ ExpressionNode nestedResourceNameExpression = parsePrimary();
skipSpace();
- if (c != '"' && c != '\'') {
- throw parseException("#parse only supported with string literal argument");
- }
- ExpressionNode nestedResourceNameExpression = parseStringLiteral(c, false);
- String nestedResourceName = nestedResourceNameExpression.evaluate(null).toString();
expect(')');
- try (Reader nestedReader = resourceOpener.openResource(nestedResourceName)) {
- Parser nestedParser = new Parser(nestedReader, nestedResourceName, resourceOpener);
- ImmutableList<Node> nestedTokens = nestedParser.parseTokens();
- return new NestedTokenNode(nestedResourceName, nestedTokens);
- }
+ return new ParseNode(
+ resourceName, startLine, nestedResourceNameExpression, resourceOpener, parseCache);
}
/**
- * Parses a {@code #macro} token from the reader. <pre>{@code
- * <macro-token> -> #macro ( <id> <macro-parameter-list> )
- * <macro-parameter-list> -> <empty> |
- * $<id> <macro-parameter-list>
+ * Parses a {@code #macro} token from the reader.
+ *
+ * <pre>{@code
+ * #macro ( <id> $<param1> $<param2> <...>) <body> #end
* }</pre>
*
* <p>Macro parameters are optionally separated by commas.
*/
private Node parseMacroDefinition() throws IOException {
+ int startLine = lineNumber();
expect('(');
skipSpace();
String name = parseId("Macro name");
@@ -462,28 +592,62 @@ class Parser {
next();
parameterNames.add(parseId("Macro parameter name"));
}
- return new MacroDefinitionTokenNode(resourceName, lineNumber(), name, parameterNames.build());
+ ParseResult parsedBody =
+ skipNewlineAndParseToStop(END_CLASS, () -> "parsing #macro starting on line " + startLine);
+ if (!macros.containsKey(name)) {
+ ImmutableList<Node> bodyNodes =
+ ImmutableList.copyOf(SetSpacing.removeInitialSpaceBeforeSet(parsedBody.nodes));
+ Node body = Node.cons(resourceName, startLine, bodyNodes);
+ Macro macro = new Macro(startLine, name, parameterNames.build(), body);
+ macros.put(name, macro);
+ }
+ return Node.emptyNode(resourceName, lineNumber());
}
+ /**
+ * {@code #directives} that Velocity supports but we currently don't, and that don't have to be
+ * followed by {@code (}. If we see one of these, we should complain, rather than just ignoring it
+ * the way we would for {@code #random} or whatever. If it <i>does</i> have to be followed by
+ * {@code (} then we will treat it as an undefined macro, which is fine.
+ */
+ private static final ImmutableSet<String> UNSUPPORTED_VELOCITY_DIRECTIVES =
+ ImmutableSet.of("stop");
+
/**
* Parses an identifier after {@code #} that is not one of the standard directives. The assumption
* is that it is a call of a macro that is defined in the template. Macro definitions are
- * extracted from the template during the second parsing phase (and not during evaluation of the
- * template as you might expect). This means that a macro can be called before it is defined.
+ * extracted from the template during parsing (and not during evaluation of the template as you
+ * might expect). This means that a macro can be called before it is defined.
+ *
* <pre>{@code
- * <macro-call> -> # <id> ( <expression-list> )
- * <expression-list> -> <empty> |
- * <expression> <optional-comma> <expression-list>
- * <optional-comma> -> <empty> | ,
+ * #<id> ()
+ * #<id> ( <expr1> )
+ * #<id> ( <expr1> <expr2>)
+ * #<id> ( <expr1> , <expr2>)
+ * ...
* }</pre>
*/
- private Node parsePossibleMacroCall(String directive) throws IOException {
- skipSpace();
+ private Node parseMacroCall(String prefix, String directive) throws IOException {
+ int startLine = lineNumber();
+ StringBuilder sb = new StringBuilder(prefix).append(directive);
+ while (Character.isWhitespace(c)) {
+ sb.appendCodePoint(c);
+ next();
+ }
if (c != '(') {
- throw parseException("Unrecognized directive #" + directive);
+ if (UNSUPPORTED_VELOCITY_DIRECTIVES.contains(directive)) {
+ throw parseException("EscapeVelocity does not currently support #" + directive);
+ }
+ // Velocity allows #foo, where #foo is not a directive and is not followed by `(` (so it can't
+ // be a macro call). Then it is just plain text. BUT, sometimes but not always, Velocity will
+ // reject #endfoo, a string beginning with #end. So we do always reject that.
+ if (directive.startsWith("end")) {
+ throw parseException("Unrecognized directive #" + directive);
+ }
+ return parsePlainText(sb);
}
next();
- ImmutableList.Builder<Node> parameterNodes = ImmutableList.builder();
+ ImmutableList.Builder<ExpressionNode> parameterNodes = ImmutableList.builder();
while (true) {
skipSpace();
if (c == ')') {
@@ -497,12 +661,31 @@ class Parser {
next();
}
}
+ Node bodyContent;
+ if (prefix.equals("#")) {
+ bodyContent = null;
+ } else {
+ ParseResult parseResult =
+ skipNewlineAndParseToStop(
+ END_CLASS, () -> "#@" + directive + " starting on line " + startLine);
+ bodyContent = Node.cons(resourceName, startLine, parseResult.nodes);
+ }
return new DirectiveNode.MacroCallNode(
- resourceName, lineNumber(), directive, parameterNodes.build());
+ resourceName, lineNumber(), directive, parameterNodes.build(), bodyContent);
+ }
+
+ private Node parseMacroCallWithBody() throws IOException {
+ assert c == '@';
+ next();
+ if (!isAsciiLetter(c)) {
+ return parsePlainText("#@");
+ }
+ String id = parseId("#@");
+ return parseMacroCall("#@", id);
}
/**
- * Parses and discards a line comment, which is {@code ##} followed by any number of characters
+ * Parses a line comment, which is {@code ##} followed by any number of characters
* up to and including the next newline.
*/
private Node parseLineComment() throws IOException {
@@ -511,11 +694,11 @@ class Parser {
next();
}
next();
- return new CommentTokenNode(resourceName, lineNumber);
+ return new CommentNode(resourceName, lineNumber);
}
/**
- * Parses and discards a block comment, which is {@code #*} followed by everything up to and
+ * Parses a block comment, which is {@code #*} followed by everything up to and
* including the next {@code *#}.
*/
private Node parseBlockComment() throws IOException {
@@ -529,7 +712,21 @@ class Parser {
next();
}
next(); // this may read EOF twice, which works
- return new CommentTokenNode(resourceName, startLine);
+ return new CommentNode(resourceName, startLine);
+ }
+
+ /**
+ * A node in the parse tree representing a comment. The only reason for recording comment nodes is
+ * so that we can skip space between a comment and a following {@code #set}, to be compatible with
+ * Velocity behaviour.
+ */
+ static class CommentNode extends Node {
+ CommentNode(String resourceName, int lineNumber) {
+ super(resourceName, lineNumber);
+ }
+
+ @Override
+ void render(EvaluationContext context, StringBuilder output) {}
}
/**
@@ -543,6 +740,15 @@ class Parser {
return parsePlainText(sb);
}
+ /**
+ * Parses plain text, which is text that contains neither {@code $} nor {@code #}. The given
+ * {@code initialChars} are the first characters of the plain text, and {@link #c} is the
+ * character after those.
+ */
+ private Node parsePlainText(String initialChars) throws IOException {
+ return parsePlainText(new StringBuilder(initialChars));
+ }
+
private Node parsePlainText(StringBuilder sb) throws IOException {
literal:
while (true) {
@@ -567,40 +773,54 @@ class Parser {
* {@code ${x}y} is a reference to the variable {@code $x}, followed by the plain text {@code y}.
* Of course {@code $xy} would be a reference to the variable {@code $xy}.
* <pre>{@code
- * <reference> -> $<reference-no-brace> |
- * ${<reference-no-brace>}
+ * <reference> -> $<maybe-silent><reference-no-brace> |
+ * $<maybe-silent>{<reference-no-brace>}
+ * <maybe-silent> -> <empty> | !
* }</pre>
*
- * <p>On entry to this method, {@link #c} is the character immediately after the {@code $}.
+ * <p>On entry to this method, {@link #c} is the character immediately after the {@code $}, or
+ * the {@code !} if there is one.
+ *
+ * @param silent true if this is {@code $!}.
*/
- private Node parseReference() throws IOException {
+ private Node parseReference(boolean silent) throws IOException {
if (c == '{') {
next();
if (!isAsciiLetter(c)) {
- return parsePlainText(new StringBuilder("${"));
+ if (silent) {
+ return parsePlainText("$!{");
+ } else {
+ return parsePlainText("${");
+ }
}
- ReferenceNode node = parseReferenceNoBrace();
+ ReferenceNode node = parseReferenceNoBrace(silent);
expect('}');
return node;
} else {
- return parseReferenceNoBrace();
+ return parseReferenceNoBrace(silent);
}
}
/**
- * Same as {@link #parseReference()}, except it really must be a reference. A {@code $} in
+ * Same as {@link #parseReference}, except it really must be a reference. A {@code $} in
* normal text doesn't start a reference if it is not followed by an identifier. But in an
* expression, for example in {@code #if ($x == 23)}, {@code $} must be followed by an
* identifier.
+ *
+ * <p>Velocity allows the {@code $!} syntax in these contexts, but it doesn't have any effect
+ * since null values are allowed anyway.
*/
private ReferenceNode parseRequiredReference() throws IOException {
+ if (c == '!') {
+ next();
+ }
if (c == '{') {
next();
- ReferenceNode node = parseReferenceNoBrace();
+ ReferenceNode node = parseReferenceNoBrace(/* silent= */ false);
expect('}');
return node;
} else {
- return parseReferenceNoBrace();
+ return parseReferenceNoBrace(/* silent= */ false);
}
}
@@ -610,10 +830,10 @@ class Parser {
* <reference-no-brace> -> <id><reference-suffix>
* }</pre>
*/
- private ReferenceNode parseReferenceNoBrace() throws IOException {
+ private ReferenceNode parseReferenceNoBrace(boolean silent) throws IOException {
String id = parseId("Reference");
- ReferenceNode lhs = new PlainReferenceNode(resourceName, lineNumber(), id);
- return parseReferenceSuffix(lhs);
+ ReferenceNode lhs = new PlainReferenceNode(resourceName, lineNumber(), id, silent);
+ return parseReferenceSuffix(lhs, silent);
}
/**
@@ -625,14 +845,14 @@ class Parser {
* }</pre>
*
* @param lhs the reference node representing the first part of the reference
- * {@code $x} in {@code $x.foo} or {@code $x.foo()}, or later {@code $x.y} in {@code $x.y.z}.
+ * {@code $x} in {@code $x.foo} or {@code $x.foo()}, or later {@code $x.y} in {@code $x.y.z}.
*/
- private ReferenceNode parseReferenceSuffix(ReferenceNode lhs) throws IOException {
+ private ReferenceNode parseReferenceSuffix(ReferenceNode lhs, boolean silent) throws IOException {
switch (c) {
case '.':
- return parseReferenceMember(lhs);
+ return parseReferenceMember(lhs, silent);
case '[':
- return parseReferenceIndex(lhs);
+ return parseReferenceIndex(lhs, silent);
default:
return lhs;
}
@@ -650,7 +870,7 @@ class Parser {
* @param lhs the reference node representing what appears to the left of the dot, like the
* {@code $x} in {@code $x.foo} or {@code $x.foo()}.
*/
- private ReferenceNode parseReferenceMember(ReferenceNode lhs) throws IOException {
+ private ReferenceNode parseReferenceMember(ReferenceNode lhs, boolean silent) throws IOException {
assert c == '.';
next();
if (!isAsciiLetter(c)) {
@@ -661,11 +881,11 @@ class Parser {
String id = parseId("Member");
ReferenceNode reference;
if (c == '(') {
- reference = parseReferenceMethodParams(lhs, id);
+ reference = parseReferenceMethodParams(lhs, id, silent);
} else {
- reference = new MemberReferenceNode(lhs, id);
+ reference = new MemberReferenceNode(lhs, id, silent);
}
- return parseReferenceSuffix(reference);
+ return parseReferenceSuffix(reference, silent);
}
/**
@@ -673,23 +893,23 @@ class Parser {
* <pre>{@code
* <method-parameter-list> -> <empty> |
* <non-empty-method-parameter-list>
- * <non-empty-method-parameter-list> -> <expression> |
- * <expression> , <non-empty-method-parameter-list>
+ * <non-empty-method-parameter-list> -> <primary> |
+ * <primary> , <non-empty-method-parameter-list>
* }</pre>
*
* @param lhs the reference node representing what appears to the left of the dot, like the
* {@code $x} in {@code $x.foo()}.
*/
- private ReferenceNode parseReferenceMethodParams(ReferenceNode lhs, String id)
+ private ReferenceNode parseReferenceMethodParams(ReferenceNode lhs, String id, boolean silent)
throws IOException {
assert c == '(';
nextNonSpace();
ImmutableList.Builder<ExpressionNode> args = ImmutableList.builder();
if (c != ')') {
- args.add(parseExpression());
+ args.add(parsePrimary(/* nullAllowed= */ true));
while (c == ',') {
nextNonSpace();
- args.add(parseExpression());
+ args.add(parsePrimary(/* nullAllowed= */ true));
}
if (c != ')') {
throw parseException("Expected )");
@@ -697,28 +917,28 @@ class Parser {
}
assert c == ')';
next();
- return new MethodReferenceNode(lhs, id, args.build());
+ return new MethodReferenceNode(lhs, id, args.build(), silent);
}
/**
- * Parses an index suffix to a method, like {@code $x[$i]}.
+ * Parses an index suffix to a reference, like {@code $x[$i]}.
* <pre>{@code
- * <reference-index> -> [ <expression> ]
+ * <reference-index> -> [ <primary> ]
* }</pre>
*
* @param lhs the reference node representing what appears to the left of the dot, like the
* {@code $x} in {@code $x[$i]}.
*/
- private ReferenceNode parseReferenceIndex(ReferenceNode lhs) throws IOException {
+ private ReferenceNode parseReferenceIndex(ReferenceNode lhs, boolean silent) throws IOException {
assert c == '[';
next();
- ExpressionNode index = parseExpression();
+ ExpressionNode index = parsePrimary();
if (c != ']') {
throw parseException("Expected ]");
}
next();
- ReferenceNode reference = new IndexReferenceNode(lhs, index);
- return parseReferenceSuffix(reference);
+ ReferenceNode reference = new IndexReferenceNode(lhs, index, silent);
+ return parseReferenceSuffix(reference, silent);
}
enum Operator {
@@ -754,6 +974,12 @@ class Parser {
public String toString() {
return symbol;
}
+
+ /** True if this is an inequality operator, one of {@code < > <= >=}. */
+ boolean isInequality() {
+ // Slightly hacky way to check.
+ return precedence == 4;
+ }
}
/**
@@ -772,14 +998,15 @@ class Parser {
}
/**
- * Parses an expression, which can occur within a directive like {@code #if} or {@code #set},
- * or within a reference like {@code $x[$a + $b]} or {@code $x.m($a + $b)}.
+ * Parses an expression, which can occur within a directive like {@code #if} or {@code #set}.
+ * Arbitrary expressions <i>can't</i> appear within a reference like {@code $x[$a + $b]} or
+ * {@code $x.m($a + $b)}, consistent with Velocity.
* <pre>{@code
* <expression> -> <and-expression> |
* <expression> || <and-expression>
* <and-expression> -> <relational-expression> |
* <and-expression> && <relational-expression>
- * <equality-exression> -> <relational-expression> |
+ * <equality-expression> -> <relational-expression> |
* <equality-expression> <equality-op> <relational-expression>
* <equality-op> -> == | !=
* <relational-expression> -> <additive-expression> |
@@ -839,6 +1066,15 @@ class Parser {
*/
private void nextOperator() throws IOException {
skipSpace();
+ switch (c) {
+ case 'a':
+ wordOperator("and", Operator.AND);
+ return;
+ case 'o':
+ wordOperator("or", Operator.OR);
+ return;
+ default: // this will fail later, but just stopping the expression here is fine
+ }
ImmutableList<Operator> possibleOperators = CODE_POINT_TO_OPERATORS.get(c);
if (possibleOperators.isEmpty()) {
currentOperator = Operator.STOP;
@@ -862,6 +1098,15 @@ class Parser {
}
currentOperator = operator;
}
+
+ private void wordOperator(String symbol, Operator operator) throws IOException {
+ String id = parseId("");
+ if (id.equals(symbol)) {
+ currentOperator = operator;
+ } else {
+ throw parseException("Expected '" + symbol + "' but was '" + id + "'");
+ }
+ }
}
/**
@@ -891,109 +1136,286 @@ class Parser {
}
}
+
/**
* Parses an expression containing only literals or references.
* <pre>{@code
* <primary> -> <reference> |
* <string-literal> |
* <integer-literal> |
- * <boolean-literal>
+ * <boolean-literal> |
+ * <list-literal>
* }</pre>
*/
private ExpressionNode parsePrimary() throws IOException {
+ return parsePrimary(false);
+ }
+
+ private ExpressionNode parsePrimary(boolean nullAllowed) throws IOException {
+ skipSpace();
ExpressionNode node;
if (c == '$') {
next();
node = parseRequiredReference();
} else if (c == '"') {
- node = parseStringLiteral(c, true);
+ node = parseStringLiteral('"', true);
} else if (c == '\'') {
- node = parseStringLiteral(c, false);
+ node = parseStringLiteral('\'', false);
} else if (c == '-') {
// Velocity does not have a negation operator. If we see '-' it must be the start of a
// negative integer literal.
next();
node = parseIntLiteral("-");
+ } else if (c == '[') {
+ node = parseListLiteral();
} else if (isAsciiDigit(c)) {
node = parseIntLiteral("");
} else if (isAsciiLetter(c)) {
- node = parseBooleanLiteral();
+ node = parseNotOrBooleanOrNullLiteral(nullAllowed);
} else {
- throw parseException("Expected an expression");
+ throw parseException("Expected a reference or a literal");
}
skipSpace();
return node;
}
/**
- * Parses a string literal, which may contain references to be expanded. Examples are
- * {@code "foo"} or {@code "foo${bar}baz"}.
+ * Parses a list or range literal.
+ *
* <pre>{@code
- * <string-literal> -> <double-quote-literal> | <single-quote-literal>
- * <double-quote-literal> -> " <double-quote-string-contents> "
- * <double-quote-string-contents> -> <empty> |
- * <reference> <double-quote-string-contents> |
- * <character-other-than-"> <double-quote-string-contents>
- * <single-quote-literal> -> ' <single-quote-string-contents> '
- * <single-quote-string-contents> -> <empty> |
- * <character-other-than-'> <single-quote-string-contents>
+ * <list-literal> -> <empty-list> | <non-empty-list>
+ * <empty-list> -> [ ]
+ * <non-empty-list> -> [ <primary> <list-end>
+ * <list-end> -> <range-end> | <remainder-of-list-literal>
+ * <range-end> -> .. <primary> ]
+ * <remainder-of-list-literal> -> <end-of-list> | , <primary> <remainder-of-list-literal>
+ * <end-of-list> -> ]
* }</pre>
*/
- private ExpressionNode parseStringLiteral(int quote, boolean allowReferences)
- throws IOException {
+ private ExpressionNode parseListLiteral() throws IOException {
+ assert c == '[';
+ nextNonSpace();
+ if (c == ']') {
+ next();
+ return new ListLiteralNode(resourceName, lineNumber(), ImmutableList.of());
+ }
+ ExpressionNode first = parsePrimary(false);
+ if (c == '.') {
+ return parseRangeLiteral(first);
+ } else {
+ return parseRemainderOfListLiteral(first);
+ }
+ }
+
+ private ExpressionNode parseRangeLiteral(ExpressionNode first) throws IOException {
+ assert c == '.';
+ next();
+ if (c != '.') {
+ throw parseException("Expected two dots (..) not just one");
+ }
+ nextNonSpace();
+ ExpressionNode last = parsePrimary(false);
+ if (c != ']') {
+ throw parseException("Expected ] at end of range literal");
+ }
+ nextNonSpace();
+ return new RangeLiteralNode(resourceName, lineNumber(), first, last);
+ }
+
+ private ExpressionNode parseRemainderOfListLiteral(ExpressionNode first) throws IOException {
+ ImmutableList.Builder<ExpressionNode> builder = ImmutableList.builder();
+ builder.add(first);
+ while (c == ',') {
+ next();
+ builder.add(parsePrimary(false));
+ }
+ if (c != ']') {
+ throw parseException("Expected ] at end of list literal");
+ }
+ next();
+ return new ListLiteralNode(resourceName, lineNumber(), builder.build());
+ }
+
+ private static class RangeLiteralNode extends ExpressionNode {
+ private final ExpressionNode first;
+ private final ExpressionNode last;
+
+ RangeLiteralNode(
+ String resourceName, int lineNumber, ExpressionNode first, ExpressionNode last) {
+ super(resourceName, lineNumber);
+ this.first = first;
+ this.last = last;
+ }
+
+ @Override
+ public String toString() {
+ return "[" + first + ".." + last + "]";
+ }
+
+ @Override
+ Object evaluate(EvaluationContext context, boolean undefinedIsFalse) {
+ int from = first.intValue(context);
+ int to = last.intValue(context);
+ ImmutableSortedSet<Integer> set =
+ (from <= to)
+ ? ContiguousSet.closed(from, to)
+ : ContiguousSet.closed(to, from).descendingSet();
+ return new ForwardingSortedSet<Integer>() {
+ @Override
+ protected ImmutableSortedSet<Integer> delegate() {
+ return set;
+ }
+
+ @Override
+ public String toString() {
+ // ContiguousSet returns [1..3] whereas Velocity uses [1, 2, 3].
+ return set.asList().toString();
+ }
+ };
+ }
+ }
+
+ private static class ListLiteralNode extends ExpressionNode {
+ private final ImmutableList<ExpressionNode> elements;
+
+ ListLiteralNode(String resourceName, int lineNumber, ImmutableList<ExpressionNode> elements) {
+ super(resourceName, lineNumber);
+ this.elements = elements;
+ }
+
+ @Override
+ public String toString() {
+ return "[" + Joiner.on(", ").join(elements) + "]";
+ }
+
+ @Override
+ Object evaluate(EvaluationContext context, boolean undefinedIsFalse) {
+ // We can't use ImmutableList because there can be nulls.
+ List<Object> list = new ArrayList<>();
+ for (ExpressionNode element : elements) {
+ list.add(element.evaluate(context));
+ }
+ return Collections.unmodifiableList(list);
+ }
+ }
+
+ /**
+ * Parses a string literal, which may contain template text to be expanded. Examples are
+ * {@code 'foo}, {@code "foo"}, and {@code "foo${bar}baz"}. Double-quote string literals
+ * ({@code expand = true}) can have arbitrary template constructs inside them, such as references,
+ * directives like {@code #if}, and macro calls. Single-quote literals really are literal.
+ */
+ private ExpressionNode parseStringLiteral(char quote, boolean expand) throws IOException {
assert c == quote;
next();
- ImmutableList.Builder<Node> nodes = ImmutableList.builder();
StringBuilder sb = new StringBuilder();
while (c != quote) {
switch (c) {
- case '\n':
case EOF:
throw parseException("Unterminated string constant");
case '\\':
throw parseException(
"Escapes in string constants are not currently supported");
- case '$':
- if (allowReferences) {
- if (sb.length() > 0) {
- nodes.add(new ConstantExpressionNode(resourceName, lineNumber(), sb.toString()));
- sb.setLength(0);
- }
- next();
- nodes.add(parseReference());
- break;
- }
- // fall through
default:
sb.appendCodePoint(c);
next();
}
}
next();
- if (sb.length() > 0) {
- nodes.add(new ConstantExpressionNode(resourceName, lineNumber(), sb.toString()));
+ String s = sb.toString();
+ ImmutableList<Node> nodes;
+ if (expand) {
+ // This is potentially something like "foo${bar}baz" or "foo#macro($bar)baz", where the text
+ // inside "..." is expanded like a mini-template. Of course it might also just be a plain old
+ // string like "foo", in which case we will just parse a single ConstantExpressionNode here.
+ String where = "string " + ParseException.where(resourceName, lineNumber());
+ Parser stringParser = new Parser(new StringReader(s), where, resourceOpener, parseCache);
+ ParseResult parseResult = stringParser.parseToStop(EOF_CLASS, () -> "outside any construct");
+ nodes = parseResult.nodes;
+ } else {
+ nodes = ImmutableList.of(new ConstantExpressionNode(resourceName, lineNumber(), s));
}
- return new StringLiteralNode(resourceName, lineNumber(), nodes.build());
+ return new StringLiteralNode(resourceName, lineNumber(), quote, nodes);
}
private static class StringLiteralNode extends ExpressionNode {
+ private final char quote;
private final ImmutableList<Node> nodes;
- StringLiteralNode(String resourceName, int lineNumber, ImmutableList<Node> nodes) {
+ StringLiteralNode(String resourceName, int lineNumber, char quote, ImmutableList<Node> nodes) {
super(resourceName, lineNumber);
+ this.quote = quote;
this.nodes = nodes;
}
@Override
- Object evaluate(EvaluationContext context) {
+ public String toString() {
+ return quote + Joiner.on("").join(nodes) + quote;
+ }
+
+ @Override
+ Object evaluate(EvaluationContext context, boolean undefinedIsFalse) {
StringBuilder sb = new StringBuilder();
for (Node node : nodes) {
- sb.append(node.evaluate(context));
+ node.render(context, sb);
}
return sb.toString();
}
}
+ /**
+ * Parses an {@code #evaluate} token from the reader.
+ *
+ * <pre>{@code
+ * #evaluate ( <primary> )
+ * }</pre>
+ */
+ private Node parseEvaluate() throws IOException {
+ int startLine = lineNumber();
+ expect('(');
+ ExpressionNode expression = parsePrimary();
+ expect(')');
+ if (c == '\n') {
+ next();
+ }
+ return new EvaluateNode(resourceName, startLine, expression);
+ }
+
+ /**
+ * An {@code #evaluate} directive. When we encounter {@code #evaluate (<foo>)}, we determine the
+ * value of {@code <foo>}, which must be a string, then we parse that string as a template and
+ * evaluate it.
+ */
+ private class EvaluateNode extends Node {
+ private final ExpressionNode expression;
+
+ EvaluateNode(String resourceName, int lineNumber, ExpressionNode expression) {
+ super(resourceName, lineNumber);
+ this.expression = expression;
+ }
+
+ @Override
+ void render(EvaluationContext context, StringBuilder sb) {
+ Object valueObject = expression.evaluate(context);
+ if (valueObject == null) { // Velocity ignores an #evaluate with a null argument.
+ return;
+ }
+ if (!(valueObject instanceof String)) {
+ throw evaluationException("Argument to #evaluate must be a string: " + valueObject);
+ }
+ String value = (String) valueObject;
+ String where = "#evaluate " + ParseException.where(resourceName, lineNumber());
+ Template template;
+ try {
+ Parser parser = new Parser(new StringReader(value), where, resourceOpener, parseCache);
+ template = parser.parse();
+ } catch (IOException e) {
+ throw evaluationException(e);
+ }
+ template.render(context, sb);
+ }
+ }
+
private ExpressionNode parseIntLiteral(String prefix) throws IOException {
StringBuilder sb = new StringBuilder(prefix);
while (isAsciiDigit(c)) {
@@ -1008,19 +1430,32 @@ class Parser {
}
/**
- * Parses a boolean literal, either {@code true} or {@code false}.
- * <boolean-literal> -> true |
- * false
+ * Parses a boolean literal, either {@code true} or {@code false}. Also allows {@code null}, but
+ * only if {@code nullAllowed} is true. Velocity allows {@code null} as a method parameter but not
+ * anywhere else.
*/
- private ExpressionNode parseBooleanLiteral() throws IOException {
- String s = parseId("Identifier without $");
- boolean value;
- if (s.equals("true")) {
- value = true;
- } else if (s.equals("false")) {
- value = false;
- } else {
- throw parseException("Identifier in expression must be preceded by $ or be true or false");
+ private ExpressionNode parseNotOrBooleanOrNullLiteral(boolean nullAllowed) throws IOException {
+ String id = parseId("Identifier without $");
+ Object value;
+ switch (id) {
+ case "true":
+ value = true;
+ break;
+ case "false":
+ value = false;
+ break;
+ case "not":
+ return new NotExpressionNode(parseUnaryExpression());
+ case "null":
+ if (nullAllowed) {
+ value = null;
+ break;
+ }
+ // fall through...
+ default:
+ String suffix = nullAllowed ? " or null" : "";
+ throw parseException(
+ "Identifier must be preceded by $ or be true or false" + suffix + ": " + id);
}
return new ConstantExpressionNode(resourceName, lineNumber(), value);
}
@@ -1075,6 +1510,7 @@ class Parser {
* including information about where it occurred.
*/
private ParseException parseException(String message) throws IOException {
+ int line = lineNumber();
StringBuilder context = new StringBuilder();
if (c == EOF) {
context.append("EOF");
@@ -1089,6 +1525,6 @@ class Parser {
context.append("...");
}
}
- return new ParseException(message, resourceName, lineNumber(), context.toString());
+ return new ParseException(message, resourceName, line, context.toString());
}
}
diff --git a/src/main/java/com/google/escapevelocity/ReferenceNode.java b/src/main/java/com/google/escapevelocity/ReferenceNode.java
index 622388f..4665d2d 100644
--- a/src/main/java/com/google/escapevelocity/ReferenceNode.java
+++ b/src/main/java/com/google/escapevelocity/ReferenceNode.java
@@ -35,8 +35,34 @@ import java.util.Optional;
* @author emcmanus@google.com (Éamonn McManus)
*/
abstract class ReferenceNode extends ExpressionNode {
- ReferenceNode(String resourceName, int lineNumber) {
+ final boolean silent;
+
+ ReferenceNode(String resourceName, int lineNumber, boolean silent) {
super(resourceName, lineNumber);
+ this.silent = silent;
+ }
+
+ @Override boolean isSilent() {
+ return silent;
+ }
+
+ EvaluationException evaluationExceptionInThis(String message) {
+ return evaluationException("In " + this + ": " + message);
+ }
+
+ /**
+ * Evaluates the first part of a complex reference, for example {@code $foo} in {@code $foo.bar}.
+ * It must not be null, and it must not be a macro's {@code $bodyContent} or the result of a
+ * {@code #define}.
+ */
+ Object evaluateLhs(ReferenceNode lhs, EvaluationContext context) {
+ Object lhsValue = lhs.evaluate(context);
+ if (lhsValue == null) {
+ throw evaluationExceptionInThis(lhs + " must not be null");
+ } else if (lhsValue instanceof Node) {
+ throw evaluationExceptionInThis(lhs + " comes from #define or is a macro's $bodyContent");
+ }
+ return lhsValue;
}
/**
@@ -46,25 +72,22 @@ abstract class ReferenceNode extends ExpressionNode {
static class PlainReferenceNode extends ReferenceNode {
final String id;
- PlainReferenceNode(String resourceName, int lineNumber, String id) {
- super(resourceName, lineNumber);
+ PlainReferenceNode(String resourceName, int lineNumber, String id, boolean silent) {
+ super(resourceName, lineNumber, silent);
this.id = id;
}
- @Override Object evaluate(EvaluationContext context) {
- if (context.varIsDefined(id)) {
- return context.getVar(id);
- } else {
- throw evaluationException("Undefined reference $" + id);
- }
+ @Override public String toString() {
+ return "$" + id;
}
- @Override
- boolean isDefinedAndTrue(EvaluationContext context) {
+ @Override Object evaluate(EvaluationContext context, boolean undefinedIsFalse) {
if (context.varIsDefined(id)) {
- return isTrue(context);
- } else {
+ return context.getVar(id);
+ } else if (undefinedIsFalse) {
return false;
+ } else {
+ throw evaluationException("Undefined reference " + this);
}
}
}
@@ -77,8 +100,8 @@ abstract class ReferenceNode extends ExpressionNode {
final ReferenceNode lhs;
final String id;
- MemberReferenceNode(ReferenceNode lhs, String id) {
- super(lhs.resourceName, lhs.lineNumber);
+ MemberReferenceNode(ReferenceNode lhs, String id, boolean silent) {
+ super(lhs.resourceName, lhs.lineNumber, silent);
this.lhs = lhs;
this.id = id;
}
@@ -86,11 +109,14 @@ abstract class ReferenceNode extends ExpressionNode {
private static final String[] PREFIXES = {"get", "is"};
private static final boolean[] CHANGE_CASE = {false, true};
- @Override Object evaluate(EvaluationContext context) {
- Object lhsValue = lhs.evaluate(context);
- if (lhsValue == null) {
- throw evaluationException("Cannot get member " + id + " of null value");
- }
+ @Override public String toString() {
+ return lhs + "." + id;
+ }
+
+ @Override Object evaluate(EvaluationContext context, boolean undefinedIsFalse) {
+ // We don't propagate undefinedIsFalse because we don't allow $foo.bar if $foo is undefined,
+ // even inside an #if expression.
+ Object lhsValue = evaluateLhs(lhs, context);
// If this is a Map, then Velocity looks up the property in the map.
if (lhsValue instanceof Map<?, ?>) {
Map<?, ?> map = (Map<?, ?>) lhsValue;
@@ -115,9 +141,12 @@ abstract class ReferenceNode extends ExpressionNode {
}
}
}
- throw evaluationException(
- "Member " + id + " does not correspond to a public getter of " + lhsValue
- + ", a " + lhsValue.getClass().getName());
+ throw evaluationExceptionInThis(
+ id
+ + " does not correspond to a public getter of "
+ + lhsValue
+ + ", a "
+ + lhsValue.getClass().getName());
}
private static String changeInitialCase(String id) {
@@ -141,27 +170,41 @@ abstract class ReferenceNode extends ExpressionNode {
final ReferenceNode lhs;
final ExpressionNode index;
- IndexReferenceNode(ReferenceNode lhs, ExpressionNode index) {
- super(lhs.resourceName, lhs.lineNumber);
+ IndexReferenceNode(ReferenceNode lhs, ExpressionNode index, boolean silent) {
+ super(lhs.resourceName, lhs.lineNumber, silent);
this.lhs = lhs;
this.index = index;
}
- @Override Object evaluate(EvaluationContext context) {
- Object lhsValue = lhs.evaluate(context);
- if (lhsValue == null) {
- throw evaluationException("Cannot index null value");
- }
+ @Override public String toString() {
+ return lhs + "[" + index + "]";
+ }
+
+ @Override Object evaluate(EvaluationContext context, boolean undefinedIsFalse) {
+ // We don't propagate undefinedIsFalse because we don't allow $foo[0] if $foo is undefined,
+ // even inside an #if expression.
+ Object lhsValue = evaluateLhs(lhs, context);
if (lhsValue instanceof List<?>) {
Object indexValue = index.evaluate(context);
if (!(indexValue instanceof Integer)) {
- throw evaluationException("List index is not an integer: " + indexValue);
+ throw evaluationExceptionInThis("list index is not an Integer: " + indexValue);
}
List<?> lhsList = (List<?>) lhsValue;
int i = (Integer) indexValue;
- if (i < 0 || i >= lhsList.size()) {
- throw evaluationException(
- "List index " + i + " is not valid for list of size " + lhsList.size());
+ if (i < 0) {
+ int newI = lhsList.size() + i;
+ if (newI < 0) {
+ throw evaluationExceptionInThis(
+ "negative list index "
+ + i
+ + " counts from the end of the list, but the list size is only "
+ + lhsList.size());
+ }
+ i = newI;
+ }
+ if (i >= lhsList.size()) {
+ throw evaluationExceptionInThis(
+ "list index " + i + " is not valid for list of size " + lhsList.size());
}
return lhsList.get(i);
} else if (lhsValue instanceof Map<?, ?>) {
@@ -171,7 +214,8 @@ abstract class ReferenceNode extends ExpressionNode {
} else {
// In general, $x[$y] is equivalent to $x.get($y). We've covered the most common cases
// above, but for other cases like Multimap we resort to evaluating the equivalent form.
- MethodReferenceNode node = new MethodReferenceNode(lhs, "get", ImmutableList.of(index));
+ MethodReferenceNode node =
+ new MethodReferenceNode(lhs, "get", ImmutableList.of(index), silent);
return node.evaluate(context);
}
}
@@ -185,13 +229,17 @@ abstract class ReferenceNode extends ExpressionNode {
final String id;
final List<ExpressionNode> args;
- MethodReferenceNode(ReferenceNode lhs, String id, List<ExpressionNode> args) {
- super(lhs.resourceName, lhs.lineNumber);
+ MethodReferenceNode(ReferenceNode lhs, String id, List<ExpressionNode> args, boolean silent) {
+ super(lhs.resourceName, lhs.lineNumber, silent);
this.lhs = lhs;
this.id = id;
this.args = args;
}
+ @Override public String toString() {
+ return lhs + "." + id + "(" + Joiner.on(", ").join(args) + ")";
+ }
+
/**
* {@inheritDoc}
*
@@ -209,11 +257,10 @@ abstract class ReferenceNode extends ExpressionNode {
* you may want to invoke a public method like {@link List#size()} on a list whose class is not
* public, such as the list returned by {@link java.util.Collections#singletonList}.
*/
- @Override Object evaluate(EvaluationContext context) {
- Object lhsValue = lhs.evaluate(context);
- if (lhsValue == null) {
- throw evaluationException("Cannot invoke method " + id + " on null value");
- }
+ @Override Object evaluate(EvaluationContext context, boolean undefinedIsFalse) {
+ // We don't propagate undefinedIsFalse because we don't allow $foo.bar() if $foo is undefined,
+ // even inside an #if expression.
+ Object lhsValue = evaluateLhs(lhs, context);
try {
return evaluate(context, lhsValue, lhsValue.getClass());
} catch (EvaluationException e) {
@@ -234,7 +281,7 @@ abstract class ReferenceNode extends ExpressionNode {
.collect(toList());
ImmutableSet<Method> publicMethodsWithName = context.publicMethodsWithName(targetClass, id);
if (publicMethodsWithName.isEmpty()) {
- throw evaluationException("No method " + id + " in " + targetClass.getName());
+ throw evaluationExceptionInThis("no method " + id + " in " + targetClass.getName());
}
List<Method> compatibleMethods = publicMethodsWithName.stream()
.filter(method -> compatibleArgs(method.getParameterTypes(), argValues))
@@ -244,15 +291,16 @@ abstract class ReferenceNode extends ExpressionNode {
compatibleMethods =
compatibleMethods.stream().filter(method -> !method.isSynthetic()).collect(toList());
}
+ // TODO(emcmanus): extract most specific overload, foo(String) rather than foo(CharSequence).
switch (compatibleMethods.size()) {
case 0:
- throw evaluationException(
- "Parameters for method " + id + " have wrong types: " + argValues);
+ throw evaluationExceptionInThis(
+ "parameters for method " + id + " have wrong types: " + argValues);
case 1:
return invokeMethod(Iterables.getOnlyElement(compatibleMethods), lhsValue, argValues);
default:
- throw evaluationException(
- "Ambiguous method invocation, could be one of:\n "
+ throw evaluationExceptionInThis(
+ "ambiguous method invocation, could be one of:\n "
+ Joiner.on("\n ").join(compatibleMethods));
}
}
diff --git a/src/main/java/com/google/escapevelocity/Reparser.java b/src/main/java/com/google/escapevelocity/Reparser.java
deleted file mode 100644
index 7f87e89..0000000
--- a/src/main/java/com/google/escapevelocity/Reparser.java
+++ /dev/null
@@ -1,283 +0,0 @@
-/*
- * Copyright (C) 2018 Google, Inc.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package com.google.escapevelocity;
-
-import static com.google.escapevelocity.Node.emptyNode;
-
-import com.google.escapevelocity.DirectiveNode.ForEachNode;
-import com.google.escapevelocity.DirectiveNode.IfNode;
-import com.google.escapevelocity.DirectiveNode.MacroCallNode;
-import com.google.escapevelocity.DirectiveNode.SetNode;
-import com.google.escapevelocity.TokenNode.CommentTokenNode;
-import com.google.escapevelocity.TokenNode.ElseIfTokenNode;
-import com.google.escapevelocity.TokenNode.ElseTokenNode;
-import com.google.escapevelocity.TokenNode.EndTokenNode;
-import com.google.escapevelocity.TokenNode.EofNode;
-import com.google.escapevelocity.TokenNode.ForEachTokenNode;
-import com.google.escapevelocity.TokenNode.IfOrElseIfTokenNode;
-import com.google.escapevelocity.TokenNode.IfTokenNode;
-import com.google.escapevelocity.TokenNode.MacroDefinitionTokenNode;
-import com.google.escapevelocity.TokenNode.NestedTokenNode;
-import com.google.common.base.CharMatcher;
-import com.google.common.collect.ImmutableList;
-import com.google.common.collect.ImmutableSet;
-import com.google.common.collect.Iterables;
-import java.util.Map;
-import java.util.Set;
-import java.util.TreeMap;
-
-/**
- * The second phase of parsing. See {@link Parser#parse()} for a description of the phases and why
- * we need them.
- *
- * @author emcmanus@google.com (Éamonn McManus)
- */
-class Reparser {
- private static final ImmutableSet<Class<? extends TokenNode>> END_SET =
- ImmutableSet.<Class<? extends TokenNode>>of(EndTokenNode.class);
- private static final ImmutableSet<Class<? extends TokenNode>> EOF_SET =
- ImmutableSet.<Class<? extends TokenNode>>of(EofNode.class);
- private static final ImmutableSet<Class<? extends TokenNode>> ELSE_ELSE_IF_END_SET =
- ImmutableSet.<Class<? extends TokenNode>>of(
- ElseTokenNode.class, ElseIfTokenNode.class, EndTokenNode.class);
-
- /**
- * The nodes that make up the input sequence. Nodes are removed one by one from this list as
- * parsing proceeds. At any time, {@link #currentNode} is the node being examined.
- */
- private final ImmutableList<Node> nodes;
-
- /**
- * The index of the node we are currently looking at while parsing.
- */
- private int nodeIndex;
-
- /**
- * Macros are removed from the input as they are found. They do not appear in the output parse
- * tree. Macro definitions are not executed in place but are all applied before template rendering
- * starts. This means that a macro can be referenced before it is defined.
- */
- private final Map<String, Macro> macros;
-
- Reparser(ImmutableList<Node> nodes) {
- this(nodes, new TreeMap<>());
- }
-
- private Reparser(ImmutableList<Node> nodes, Map<String, Macro> macros) {
- this.nodes = removeSpaceBeforeSet(nodes);
- this.nodeIndex = 0;
- this.macros = macros;
- }
-
- Template reparse() {
- Node root = reparseNodes();
- linkMacroCalls();
- return new Template(root);
- }
-
- private Node reparseNodes() {
- return parseTo(EOF_SET, new EofNode((String) null, 1));
- }
-
- /**
- * Returns a copy of the given list where spaces have been moved where appropriate after {@code
- * #set}. This hack is needed to match what appears to be special treatment in Apache Velocity of
- * spaces before {@code #set} directives. If you have <i>thing</i> <i>whitespace</i> {@code #set},
- * then the whitespace is deleted if the <i>thing</i> is a comment ({@code ##...\n}); a reference
- * ({@code $x} or {@code $x.foo} etc); a macro definition; or another {@code #set}.
- */
- private static ImmutableList<Node> removeSpaceBeforeSet(ImmutableList<Node> nodes) {
- assert Iterables.getLast(nodes) instanceof EofNode;
- // Since the last node is EofNode, the i + 1 and i + 2 accesses below are safe.
- ImmutableList.Builder<Node> newNodes = ImmutableList.builder();
- for (int i = 0; i < nodes.size(); i++) {
- Node nodeI = nodes.get(i);
- newNodes.add(nodeI);
- if (shouldDeleteSpaceBetweenThisAndSet(nodeI)
- && isWhitespaceLiteral(nodes.get(i + 1))
- && nodes.get(i + 2) instanceof SetNode) {
- // Skip the space.
- i++;
- }
- }
- return newNodes.build();
- }
-
- private static boolean shouldDeleteSpaceBetweenThisAndSet(Node node) {
- return node instanceof CommentTokenNode
- || node instanceof ReferenceNode
- || node instanceof SetNode
- || node instanceof MacroDefinitionTokenNode;
- }
-
- private static boolean isWhitespaceLiteral(Node node) {
- if (node instanceof ConstantExpressionNode) {
- Object constant = node.evaluate(null);
- return constant instanceof String && CharMatcher.whitespace().matchesAllOf((String) constant);
- }
- return false;
- }
-
- /**
- * Parse subtrees until one of the token types in {@code stopSet} is encountered.
- * If this is the top level, {@code stopSet} will include {@link EofNode} so parsing will stop
- * when it reaches the end of the input. Otherwise, if an {@code EofNode} is encountered it is an
- * error because we have something like {@code #if} without {@code #end}.
- *
- * @param stopSet the kinds of tokens that will stop the parse. For example, if we are parsing
- * after an {@code #if}, we will stop at any of {@code #else}, {@code #elseif},
- * or {@code #end}.
- * @param forWhat the token that triggered this call, for example the {@code #if} whose
- * {@code #end} etc we are looking for.
- *
- * @return a Node that is the concatenation of the parsed subtrees
- */
- private Node parseTo(Set<Class<? extends TokenNode>> stopSet, TokenNode forWhat) {
- ImmutableList.Builder<Node> nodeList = ImmutableList.builder();
- while (true) {
- Node currentNode = currentNode();
- if (stopSet.contains(currentNode.getClass())) {
- break;
- }
- if (currentNode instanceof EofNode) {
- throw new ParseException(
- "Reached end of file while parsing " + forWhat.name(),
- forWhat.resourceName,
- forWhat.lineNumber);
- }
- Node parsed;
- if (currentNode instanceof TokenNode) {
- parsed = parseTokenNode();
- } else {
- parsed = currentNode;
- nextNode();
- }
- nodeList.add(parsed);
- }
- return Node.cons(forWhat.resourceName, forWhat.lineNumber, nodeList.build());
- }
-
- private Node currentNode() {
- return nodes.get(nodeIndex);
- }
-
- private Node nextNode() {
- Node currentNode = currentNode();
- if (currentNode instanceof EofNode) {
- return currentNode;
- } else {
- nodeIndex++;
- return currentNode();
- }
- }
-
- private Node parseTokenNode() {
- TokenNode tokenNode = (TokenNode) currentNode();
- nextNode();
- if (tokenNode instanceof CommentTokenNode) {
- return emptyNode(tokenNode.resourceName, tokenNode.lineNumber);
- } else if (tokenNode instanceof IfTokenNode) {
- return parseIfOrElseIf((IfTokenNode) tokenNode);
- } else if (tokenNode instanceof ForEachTokenNode) {
- return parseForEach((ForEachTokenNode) tokenNode);
- } else if (tokenNode instanceof NestedTokenNode) {
- return parseNested((NestedTokenNode) tokenNode);
- } else if (tokenNode instanceof MacroDefinitionTokenNode) {
- return parseMacroDefinition((MacroDefinitionTokenNode) tokenNode);
- } else {
- throw new IllegalArgumentException(
- "Unexpected token: " + tokenNode.name() + " on line " + tokenNode.lineNumber);
- }
- }
-
- private Node parseForEach(ForEachTokenNode forEach) {
- Node body = parseTo(END_SET, forEach);
- nextNode(); // Skip #end
- return new ForEachNode(
- forEach.resourceName, forEach.lineNumber, forEach.var, forEach.collection, body);
- }
-
- private Node parseIfOrElseIf(IfOrElseIfTokenNode ifOrElseIf) {
- Node truePart = parseTo(ELSE_ELSE_IF_END_SET, ifOrElseIf);
- Node falsePart;
- Node token = currentNode();
- nextNode(); // Skip #else or #elseif (cond) or #end.
- if (token instanceof EndTokenNode) {
- falsePart = emptyNode(token.resourceName, token.lineNumber);
- } else if (token instanceof ElseTokenNode) {
- falsePart = parseTo(END_SET, ifOrElseIf);
- nextNode(); // Skip #end
- } else if (token instanceof ElseIfTokenNode) {
- // We've seen #if (condition1) ... #elseif (condition2). currentToken is the first token
- // after (condition2). We pretend that we've just seen #if (condition2) and parse out
- // the remainder (which might have further #elseif and final #else). Then we pretend that
- // we actually saw #if (condition1) ... #else #if (condition2) ...remainder ... #end #end.
- falsePart = parseIfOrElseIf((ElseIfTokenNode) token);
- } else {
- throw new AssertionError(currentNode());
- }
- return new IfNode(
- ifOrElseIf.resourceName, ifOrElseIf.lineNumber, ifOrElseIf.condition, truePart, falsePart);
- }
-
- // This is a #parse("foo.vm") directive. We've already done the first phase of parsing on the
- // contents of foo.vm. Now we need to do the second phase, and insert the result into the
- // reparsed nodes. We can call Reparser recursively, but we must ensure that any macros found
- // are added to the containing Reparser's macro definitions.
- private Node parseNested(NestedTokenNode nested) {
- Reparser reparser = new Reparser(nested.nodes, this.macros);
- return reparser.reparseNodes();
- }
-
- private Node parseMacroDefinition(MacroDefinitionTokenNode macroDefinition) {
- Node body = parseTo(END_SET, macroDefinition);
- nextNode(); // Skip #end
- if (!macros.containsKey(macroDefinition.name)) {
- Macro macro = new Macro(
- macroDefinition.lineNumber, macroDefinition.name, macroDefinition.parameterNames, body);
- macros.put(macroDefinition.name, macro);
- }
- return emptyNode(macroDefinition.resourceName, macroDefinition.lineNumber);
- }
-
- private void linkMacroCalls() {
- for (Node node : nodes) {
- if (node instanceof MacroCallNode) {
- linkMacroCall((MacroCallNode) node);
- }
- }
- }
-
- private void linkMacroCall(MacroCallNode macroCall) {
- Macro macro = macros.get(macroCall.name());
- if (macro == null) {
- throw new ParseException(
- "#" + macroCall.name()
- + " is neither a standard directive nor a macro that has been defined",
- macroCall.resourceName,
- macroCall.lineNumber);
- }
- if (macro.parameterCount() != macroCall.argumentCount()) {
- throw new ParseException(
- "Wrong number of arguments to #" + macroCall.name()
- + ": expected " + macro.parameterCount()
- + ", got " + macroCall.argumentCount(),
- macroCall.resourceName,
- macroCall.lineNumber);
- }
- macroCall.setMacro(macro);
- }
-}
diff --git a/src/main/java/com/google/escapevelocity/SetSpacing.java b/src/main/java/com/google/escapevelocity/SetSpacing.java
new file mode 100644
index 0000000..c162f88
--- /dev/null
+++ b/src/main/java/com/google/escapevelocity/SetSpacing.java
@@ -0,0 +1,82 @@
+/*
+ * Copyright (C) 2018 Google, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.google.escapevelocity;
+
+import com.google.common.collect.Iterables;
+import com.google.escapevelocity.DirectiveNode.SetNode;
+import com.google.escapevelocity.Parser.CommentNode;
+import java.util.List;
+
+/**
+ * Special treatment of spaces before {@code #set} directives.
+ *
+ * @author emcmanus@google.com (Éamonn McManus)
+ */
+final class SetSpacing {
+ private SetSpacing() {}
+
+ /**
+ * Implements removal of spaces where appropriate before {@code #set} directives. The last element
+ * of the given list is removed if it consists of space and if that space occurs in a context
+ * where it is removed before {@code #set}. This hack is needed to match what appears to be
+ * special treatment in Apache Velocity of spaces before {@code #set} directives. If you have
+ * <i>thing</i> <i>horizontal whitespace</i> {@code #set}, then the whitespace is deleted if the
+ * <i>thing</i> is a comment ({@code ##...\n}); a reference ({@code $x} or {@code $x.foo} etc); or
+ * another {@code #set}. Spaces are also removed before {@code #set} at the start of a macro
+ * definition, but that is implemented by calling {@link #removeInitialSpaceBeforeSet}.
+ *
+ * <p>Newlines are already deleted after a directive or comment, so space will be deleted before
+ * the second {@code #set} here:
+ *
+ * <pre>
+ * #set ($x = 17)
+ * #set ($y = 23)
+ * </pre>
+ *
+ * but not here:
+ *
+ * <pre>
+ * #set ($x = 17)
+ *
+ * #set ($y = 23)
+ * </pre>
+ */
+ static boolean shouldRemoveLastNodeBeforeSet(List<Node> nodes) {
+ if (nodes.isEmpty()) {
+ return false;
+ }
+ Node potentialSpaceBeforeSet = Iterables.getLast(nodes);
+ if (nodes.size() == 1) {
+ return potentialSpaceBeforeSet.isHorizontalWhitespace();
+ }
+ Node beforeSpace = nodes.get(nodes.size() - 2);
+ if (beforeSpace instanceof ReferenceNode
+ || beforeSpace instanceof CommentNode
+ || beforeSpace instanceof DirectiveNode) {
+ return potentialSpaceBeforeSet.isHorizontalWhitespace();
+ }
+ return false;
+ }
+
+ static List<Node> removeInitialSpaceBeforeSet(List<Node> nodes) {
+ if (nodes.size() >= 2
+ && nodes.get(0).isHorizontalWhitespace()
+ && nodes.get(1) instanceof SetNode) {
+ return nodes.subList(1, nodes.size());
+ }
+ return nodes;
+ }
+}
diff --git a/src/main/java/com/google/escapevelocity/StopNode.java b/src/main/java/com/google/escapevelocity/StopNode.java
new file mode 100644
index 0000000..e23a5a7
--- /dev/null
+++ b/src/main/java/com/google/escapevelocity/StopNode.java
@@ -0,0 +1,89 @@
+/*
+ * Copyright (C) 2018 Google, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.google.escapevelocity;
+
+/**
+ * A parsing node that represents the end of a span, such as the {@code #end} after the body of a
+ * {@code #foreach} or an {@code #else} in an {@code #if}. The end of the file is also one of these.
+ *
+ * <p>These nodes are used during parsing but do not end up in the final parse tree.
+ *
+ * @author emcmanus@google.com (Éamonn McManus)
+ */
+abstract class StopNode extends Node {
+ StopNode(String resourceName, int lineNumber) {
+ super(resourceName, lineNumber);
+ }
+
+ /**
+ * This method always throws an exception because a node like this should never be found in the
+ * final parse tree.
+ */
+ @Override
+ void render(EvaluationContext context, StringBuilder output) {
+ throw new UnsupportedOperationException(getClass().getName());
+ }
+
+ /**
+ * The name of the token, for use in parse error messages.
+ */
+ abstract String name();
+
+ /**
+ * A synthetic node that represents the end of the input. This node is the last one in the
+ * initial token string and also the last one in the parse tree.
+ */
+ static final class EofNode extends StopNode {
+ EofNode(String resourceName, int lineNumber) {
+ super(resourceName, lineNumber);
+ }
+
+ @Override
+ String name() {
+ return "end of file";
+ }
+ }
+
+ static final class EndNode extends StopNode {
+ EndNode(String resourceName, int lineNumber) {
+ super(resourceName, lineNumber);
+ }
+
+ @Override String name() {
+ return "#end";
+ }
+ }
+
+ static final class ElseIfNode extends StopNode {
+ ElseIfNode(String resourceName, int lineNumber) {
+ super(resourceName, lineNumber);
+ }
+
+ @Override String name() {
+ return "#elseif";
+ }
+ }
+
+ static final class ElseNode extends StopNode {
+ ElseNode(String resourceName, int lineNumber) {
+ super(resourceName, lineNumber);
+ }
+
+ @Override String name() {
+ return "#else";
+ }
+ }
+}
diff --git a/src/main/java/com/google/escapevelocity/Template.java b/src/main/java/com/google/escapevelocity/Template.java
index 6bc75c2..b1f4572 100644
--- a/src/main/java/com/google/escapevelocity/Template.java
+++ b/src/main/java/com/google/escapevelocity/Template.java
@@ -15,10 +15,13 @@
*/
package com.google.escapevelocity;
+import com.google.common.collect.ImmutableMap;
import com.google.escapevelocity.EvaluationContext.PlainEvaluationContext;
import java.io.IOException;
import java.io.Reader;
+import java.util.LinkedHashMap;
import java.util.Map;
+import java.util.TreeMap;
/**
* A template expressed in EscapeVelocity, a subset of the Velocity Template Language (VTL) from
@@ -31,7 +34,17 @@ import java.util.Map;
// TODO(emcmanus): spell out exactly what Velocity features are unsupported.
public class Template {
private final Node root;
-
+
+ /**
+ * Macros that are defined in this template (this exact VTL file). If the template includes
+ * {@code #parse} directives, those might end up defining other macros when a {@code #parse} is
+ * evaluated. The {@code #parse} produces a separate {@code Template} object with its own
+ * {@code macros} map. When the root {@code Template} is evaluated, the {@link EvaluationContext}
+ * starts off with the macros here, and each {@code #parse} that is executed may add macros to the
+ * map in the {@code EvaluationContext}.
+ */
+ private final ImmutableMap<String, Macro> macros;
+
/**
* Caches {@link Method} objects for public methods accessed through references. The first time
* we evaluate {@code $var.property} or {@code $var.method(...)} for a {@code $var} of a given
@@ -54,17 +67,21 @@ public class Template {
* if (inputStream == null) {
* throw new IOException("Unknown resource: " + resourceName);
* }
- * return new BufferedReader(InputStreamReader(inputStream, StandardCharsets.UTF_8));
+ * return new BufferedReader(new InputStreamReader(inputStream, StandardCharsets.UTF_8));
* };
+ * Template template = Template.parseFrom("foo.vm", resourceOpener);
* }</pre>
*/
@FunctionalInterface
public interface ResourceOpener {
/**
- * Returns a {@code Reader} that will be used to read the given resource, then closed.
+ * Returns a {@code Reader} that will be used to read the given resource, then closed. The
+ * caller of this method will perform its own buffering (via {@link java.io.BufferedReader
+ * BufferedReader}), so the returned Reader doesn't need to be buffered.
*
- * @param resourceName the name of the resource to be read. This will never be null.
+ * @param resourceName the name of the resource to be read. This can be null if {@code
+ * Template.parseFrom} is called with a null {@code resourceName}.
* @return a {@code Reader} for the resource.
* @throws IOException if the resource cannot be opened.
*/
@@ -73,14 +90,17 @@ public class Template {
/**
* Parses a VTL template from the given {@code Reader}. The template cannot reference other
- * templates (for example with {@code #parse}). For that, use
- * {@link #parseFrom(String, ResourceOpener)}.
+ * templates (for example with {@code #parse}). For that, use {@link #parseFrom(String,
+ * ResourceOpener)}.
*
* @param reader a Reader that will supply the text of the template. It will be closed on return
- * from this method.
+ * from this method. The Reader will be buffered internally by this method (via {@link
+ * java.io.BufferedReader BufferedReader}), so the passed-in Reader doesn't need to perform
+ * its own buffering.
* @return an object representing the parsed template.
* @throws IOException if there is an exception reading from {@code reader}, or if the template
* references another template via {@code #parse}.
+ * @throws ParseException if the text of the template could not be parsed.
*/
public static Template parseFrom(Reader reader) throws IOException {
ResourceOpener resourceOpener = resourceName -> {
@@ -98,23 +118,39 @@ public class Template {
}
/**
- * Parse a VTL template of the given name using the given {@code ResourceOpener}.
+ * Parses a VTL template of the given name using the given {@code ResourceOpener}.
*
* @param resourceName name of the resource. May be null.
* @param resourceOpener used to open the initial resource and resources referenced by
* {@code #parse} directives in the template.
* @return an object representing the parsed template.
* @throws IOException if there is an exception opening or reading from any resource.
+ * @throws ParseException if the text of the template could not be parsed.
*/
public static Template parseFrom(
String resourceName, ResourceOpener resourceOpener) throws IOException {
+
+ // This cache is passed into the top-level parser, and saved in the ParseNode for any #parse
+ // directive. When a #parse is evaluated, it either finds the already-parsed Template for the
+ // resource named in its argument, or it parses the resource and saves the result in this
+ // cache. If it parses the resource, it will pass in the same parseCache to the parseFrom method
+ // below so the parseCache will be shared by any #parse directives in nested templates.
+ Map<String, Template> parseCache = new TreeMap<>();
+
+ return parseFrom(resourceName, resourceOpener, parseCache);
+ }
+
+ static Template parseFrom(
+ String resourceName, ResourceOpener resourceOpener, Map<String, Template> parseCache)
+ throws IOException {
try (Reader reader = resourceOpener.openResource(resourceName)) {
- return new Parser(reader, resourceName, resourceOpener).parse();
+ return new Parser(reader, resourceName, resourceOpener, parseCache).parse();
}
}
- Template(Node root) {
+ Template(Node root, ImmutableMap<String, Macro> macros) {
this.root = root;
+ this.macros = macros;
}
/**
@@ -123,11 +159,37 @@ public class Template {
* @param vars a map where the keys are variable names and the values are the corresponding
* variable values. For example, if {@code "x"} maps to 23, then {@code $x} in the template
* will expand to 23.
- *
* @return the string result of evaluating the template.
+ * @throws EvaluationException if the evaluation failed, for example because of an undefined
+ * reference. If the template contains a {@code #parse} directive, there may be an exception
+ * such as {@link ParseException} or {@link IOException} when the nested template is read and
+ * parsed. That exception will then be the {@linkplain Throwable#getCause() cause} of an
+ * {@link EvaluationException}.
*/
public String evaluate(Map<String, ?> vars) {
- EvaluationContext evaluationContext = new PlainEvaluationContext(vars, methodFinder);
- return String.valueOf(root.evaluate(evaluationContext));
+ // This is so that a nested #parse can define new macros. Obviously that shouldn't affect the
+ // macros stored in the template, since later calls to `evaluate` should not see changes.
+ Map<String, Macro> modifiableMacros = new LinkedHashMap<>(macros);
+ EvaluationContext evaluationContext =
+ new PlainEvaluationContext(vars, modifiableMacros, methodFinder);
+ StringBuilder output = new StringBuilder(1024);
+ // The default size of 16 is going to be too small for the vast majority of rendered templates.
+ // We use a somewhat arbitrary larger starting size instead.
+ try {
+ render(evaluationContext, output);
+ } catch (BreakException e) {
+ if (e.forEachScope()) {
+ throw new EvaluationException("#break($foreach) not inside #foreach: " + e.getMessage());
+ }
+ }
+ return output.toString();
+ }
+
+ void render(EvaluationContext context, StringBuilder output) {
+ root.render(context, output);
+ }
+
+ ImmutableMap<String, Macro> getMacros() {
+ return macros;
}
}
diff --git a/src/main/java/com/google/escapevelocity/TokenNode.java b/src/main/java/com/google/escapevelocity/TokenNode.java
deleted file mode 100644
index 971ad30..0000000
--- a/src/main/java/com/google/escapevelocity/TokenNode.java
+++ /dev/null
@@ -1,169 +0,0 @@
-/*
- * Copyright (C) 2018 Google, Inc.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package com.google.escapevelocity;
-
-import com.google.common.collect.ImmutableList;
-import java.util.List;
-
-/**
- * A parsing node that will be deleted during the construction of the parse tree, to be replaced
- * by a higher-level construct such as {@link DirectiveNode.IfNode}. See {@link Parser#parse()}
- * for a description of the way these tokens work.
- *
- * @author emcmanus@google.com (Éamonn McManus)
- */
-abstract class TokenNode extends Node {
- TokenNode(String resourceName, int lineNumber) {
- super(resourceName, lineNumber);
- }
-
- /**
- * This method always throws an exception because a node like this should never be found in the
- * final parse tree.
- */
- @Override Object evaluate(EvaluationContext vars) {
- throw new UnsupportedOperationException(getClass().getName());
- }
-
- /**
- * The name of the token, for use in parse error messages.
- */
- abstract String name();
-
- /**
- * A synthetic node that represents the end of the input. This node is the last one in the
- * initial token string and also the last one in the parse tree.
- */
- static final class EofNode extends TokenNode {
- EofNode(String resourceName, int lineNumber) {
- super(resourceName, lineNumber);
- }
-
- @Override
- String name() {
- return "end of file";
- }
- }
-
- static final class EndTokenNode extends TokenNode {
- EndTokenNode(String resourceName, int lineNumber) {
- super(resourceName, lineNumber);
- }
-
- @Override String name() {
- return "#end";
- }
- }
-
- /**
- * A node in the parse tree representing a comment. Comments are introduced by {@code ##} and
- * extend to the end of the line. The only reason for recording comment nodes is so that we can
- * skip space between a comment and a following {@code #set}, to be compatible with Velocity
- * behaviour.
- */
- static class CommentTokenNode extends TokenNode {
- CommentTokenNode(String resourceName, int lineNumber) {
- super(resourceName, lineNumber);
- }
-
- @Override String name() {
- return "##";
- }
- }
-
- abstract static class IfOrElseIfTokenNode extends TokenNode {
- final ExpressionNode condition;
-
- IfOrElseIfTokenNode(ExpressionNode condition) {
- super(condition.resourceName, condition.lineNumber);
- this.condition = condition;
- }
- }
-
- static final class IfTokenNode extends IfOrElseIfTokenNode {
- IfTokenNode(ExpressionNode condition) {
- super(condition);
- }
-
- @Override String name() {
- return "#if";
- }
- }
-
- static final class ElseIfTokenNode extends IfOrElseIfTokenNode {
- ElseIfTokenNode(ExpressionNode condition) {
- super(condition);
- }
-
- @Override String name() {
- return "#elseif";
- }
- }
-
- static final class ElseTokenNode extends TokenNode {
- ElseTokenNode(String resourceName, int lineNumber) {
- super(resourceName, lineNumber);
- }
-
- @Override String name() {
- return "#else";
- }
- }
-
- static final class ForEachTokenNode extends TokenNode {
- final String var;
- final ExpressionNode collection;
-
- ForEachTokenNode(String var, ExpressionNode collection) {
- super(collection.resourceName, collection.lineNumber);
- this.var = var;
- this.collection = collection;
- }
-
- @Override String name() {
- return "#foreach";
- }
- }
-
- static final class NestedTokenNode extends TokenNode {
- final ImmutableList<Node> nodes;
-
- NestedTokenNode(String resourceName, ImmutableList<Node> nodes) {
- super(resourceName, 1);
- this.nodes = nodes;
- }
-
- @Override String name() {
- return "#parse(\"" + resourceName + "\")";
- }
- }
-
- static final class MacroDefinitionTokenNode extends TokenNode {
- final String name;
- final ImmutableList<String> parameterNames;
-
- MacroDefinitionTokenNode(
- String resourceName, int lineNumber, String name, List<String> parameterNames) {
- super(resourceName, lineNumber);
- this.name = name;
- this.parameterNames = ImmutableList.copyOf(parameterNames);
- }
-
- @Override String name() {
- return "#macro(" + name + ")";
- }
- }
-}
diff --git a/src/test/java/com/google/escapevelocity/TemplateTest.java b/src/test/java/com/google/escapevelocity/TemplateTest.java
index 0503125..f5749fe 100644
--- a/src/test/java/com/google/escapevelocity/TemplateTest.java
+++ b/src/test/java/com/google/escapevelocity/TemplateTest.java
@@ -17,36 +17,38 @@ package com.google.escapevelocity;
import static com.google.common.truth.Truth.assertThat;
import static com.google.common.truth.Truth.assertWithMessage;
+import static com.google.common.truth.TruthJUnit.assume;
+import static org.junit.Assert.assertThrows;
import static org.junit.Assert.fail;
+import com.google.common.base.Joiner;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSetMultimap;
import com.google.common.truth.Expect;
-import java.io.ByteArrayInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
-import java.io.InputStream;
import java.io.StringReader;
import java.io.StringWriter;
import java.io.UncheckedIOException;
-import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
+import java.util.Collections;
import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.LinkedList;
import java.util.Map;
+import java.util.Objects;
+import java.util.Set;
import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.function.Function;
import java.util.function.Supplier;
-import org.apache.commons.collections.ExtendedProperties;
import org.apache.velocity.VelocityContext;
-import org.apache.velocity.exception.ResourceNotFoundException;
+import org.apache.velocity.app.VelocityEngine;
import org.apache.velocity.exception.VelocityException;
import org.apache.velocity.runtime.RuntimeConstants;
-import org.apache.velocity.runtime.RuntimeInstance;
-import org.apache.velocity.runtime.log.NullLogChute;
-import org.apache.velocity.runtime.parser.node.SimpleNode;
-import org.apache.velocity.runtime.resource.Resource;
-import org.apache.velocity.runtime.resource.loader.ResourceLoader;
-import org.junit.Before;
+import org.apache.velocity.runtime.resource.loader.StringResourceLoader;
+import org.apache.velocity.runtime.resource.util.StringResourceRepository;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TestName;
@@ -61,24 +63,38 @@ public class TemplateTest {
@Rule public TestName testName = new TestName();
@Rule public Expect expect = Expect.create();
- private RuntimeInstance velocityRuntimeInstance;
+ private enum Version {V1, V2}
+ private static final Version VERSION;
- @Before
- public void initVelocityRuntimeInstance() {
- velocityRuntimeInstance = newVelocityRuntimeInstance();
- velocityRuntimeInstance.init();
+ static {
+ Version version;
+ try {
+ // The Runtime class was deprecated in v1.7 and deleted in v2.0.
+ Class.forName("org.apache.velocity.runtime.Runtime");
+ version = Version.V1;
+ } catch (ClassNotFoundException e) {
+ version = Version.V2;
+ }
+ VERSION = version;
}
- private RuntimeInstance newVelocityRuntimeInstance() {
- RuntimeInstance runtimeInstance = new RuntimeInstance();
+ private VelocityEngine newVelocityEngine() {
+ VelocityEngine engine = new VelocityEngine();
// Ensure that $undefinedvar will produce an exception rather than outputting $undefinedvar.
- runtimeInstance.setProperty(RuntimeConstants.RUNTIME_REFERENCES_STRICT, "true");
+ engine.setProperty(RuntimeConstants.RUNTIME_REFERENCES_STRICT, "true");
+
+ // Set properties to make Velocity 2.x more like 1.7.
+ engine.setProperty("directive.if.empty_check", "false");
+ engine.setProperty("parser.allow_hyphen_in_identifiers", "true");
+ engine.setProperty("parser.space_gobbling", "bc");
// Disable any logging that Velocity might otherwise see fit to do.
- runtimeInstance.setProperty(RuntimeConstants.RUNTIME_LOG_LOGSYSTEM_CLASS, new NullLogChute());
- runtimeInstance.setProperty(RuntimeConstants.RUNTIME_LOG_LOGSYSTEM, new NullLogChute());
- return runtimeInstance;
+ // This has no effect on V2, but there you can shut logging up via slf4j.
+ engine.setProperty(
+ "runtime.log.logsystem.class", "org.apache.velocity.runtime.log.NullLogChute");
+
+ return engine;
}
private void compare(String template) {
@@ -115,41 +131,45 @@ public class TemplateTest {
}
private String velocityRender(String template, Map<String, ?> vars) {
+ VelocityEngine velocityEngine = newVelocityEngine();
+ velocityEngine.init();
VelocityContext velocityContext = new VelocityContext(new TreeMap<>(vars));
StringWriter writer = new StringWriter();
- SimpleNode parsedTemplate;
- try {
- parsedTemplate = velocityRuntimeInstance.parse(
- new StringReader(template), testName.getMethodName());
- } catch (org.apache.velocity.runtime.parser.ParseException e) {
- throw new AssertionError(e);
- }
- boolean rendered = velocityRuntimeInstance.render(
- velocityContext, writer, parsedTemplate.getTemplateName(), parsedTemplate);
- assertThat(rendered).isTrue();
+ String templateName = testName.getMethodName();
+ boolean rendered =
+ velocityEngine.evaluate(velocityContext, writer, templateName, new StringReader(template));
+ assertWithMessage(templateName).that(rendered).isTrue();
return writer.toString();
}
- private void expectParseException(
+ private void expectException(
String template,
String expectedMessageSubstring) {
- Exception velocityException = null;
- try {
- SimpleNode parsedTemplate =
- velocityRuntimeInstance.parse(new StringReader(template), testName.getMethodName());
- VelocityContext velocityContext = new VelocityContext(new TreeMap<>());
- velocityRuntimeInstance.render(
- velocityContext, new StringWriter(), parsedTemplate.getTemplateName(), parsedTemplate);
- fail("Velocity did not throw an exception for this template");
- } catch (org.apache.velocity.runtime.parser.ParseException | VelocityException expected) {
- velocityException = expected;
- }
+ expectException(template, ImmutableMap.of(), expectedMessageSubstring);
+ }
+
+ private void expectException(
+ String template,
+ Map<String, ?> vars,
+ String expectedMessageSubstring) {
+ VelocityEngine velocityEngine = newVelocityEngine();
+ velocityEngine.init();
+ VelocityContext velocityContext = new VelocityContext(new TreeMap<>(vars));
+ String templateName = testName.getMethodName();
+ VelocityException velocityException =
+ assertThrows(
+ "Velocity did not throw an exception for this template",
+ VelocityException.class,
+ () ->
+ velocityEngine.evaluate(
+ velocityContext, new StringWriter(), templateName, new StringReader(template)));
try {
- Template.parseFrom(new StringReader(template));
+ Template parsedTemplate = Template.parseFrom(new StringReader(template));
+ parsedTemplate.evaluate(vars);
fail("Velocity generated an exception, but EscapeVelocity did not: " + velocityException);
} catch (IOException e) {
throw new UncheckedIOException(e);
- } catch (ParseException expected) {
+ } catch (ParseException | EvaluationException expected) {
assertWithMessage("Got expected exception, but message did not match")
.that(expected).hasMessageThat().contains(expectedMessageSubstring);
}
@@ -186,6 +206,65 @@ public class TemplateTest {
compare("${foo}#${bar}", ImmutableMap.of("foo", "xxx", "bar", "yyy"));
}
+ @Test
+ public void ignoreUnrecognizedDirective() {
+ compare("<a href=\"http://google.com/foo#bar\">");
+ compare("#bar");
+ // Using a name like #endfoo sometimes triggers an exception with Velocity. Here we check
+ // whether any other standard directive string has this behaviour, and apparently none do.
+ compare("#breakx");
+ compare("#definex");
+ compare("#elseifx");
+ compare("#elsex");
+ compare("#evaluatex");
+ compare("#foreachx");
+ compare("#ifx");
+ compare("#includex");
+ compare("#macrox");
+ compare("#parsex");
+ compare("#setx");
+ compare("#stopx");
+ expectException("#endx", "Unrecognized directive #endx");
+ }
+
+ // Since we are lax with #foo in general, make sure we don't just ignore Velocity directives that
+ // we don't support. There are two varieties: ones that must be followed by `(`, which we will
+ // treat as undefined macros; and ones that are not necessarily followed by `(`, which we handle
+ // explicitly.
+
+ @Test
+ public void unsupportedDirectives_paren() throws Exception {
+ String[] unsupportedDirectives = {
+ "#include('x.vm')",
+ };
+ for (String unsupportedDirective : unsupportedDirectives) {
+ Template template = Template.parseFrom(new StringReader(unsupportedDirective));
+ EvaluationException e =
+ assertThrows(
+ unsupportedDirective,
+ EvaluationException.class,
+ () -> template.evaluate(ImmutableMap.of()));
+ assertThat(e)
+ .hasMessageThat()
+ .contains("is neither a standard directive nor a macro");
+ }
+ }
+
+ @Test
+ public void unsupportedDirectives_noParen() {
+ String[] unsupportedDirectives = {"#stop"};
+ for (String unsupportedDirective : unsupportedDirectives) {
+ ParseException e =
+ assertThrows(
+ unsupportedDirective,
+ ParseException.class,
+ () -> Template.parseFrom(new StringReader(unsupportedDirective)));
+ assertThat(e)
+ .hasMessageThat()
+ .contains("EscapeVelocity does not currently support " + unsupportedDirective);
+ }
+ }
+
@Test
public void blockQuote() {
compare("#[[]]#");
@@ -215,6 +294,7 @@ public class TemplateTest {
// The first $ is plain text and the second one starts a reference.
compare(" $$foo ", ImmutableMap.of("foo", true));
compare(" $${foo} ", ImmutableMap.of("foo", true));
+ compare(" $!$foo ", ImmutableMap.of("foo", true));
}
@Test
@@ -232,6 +312,12 @@ public class TemplateTest {
compare("=${t.name}=", ImmutableMap.of("t", Thread.currentThread()));
}
+ @Test
+ public void braceNotFollowedById() {
+ compare("${??");
+ compare("$!{??");
+ }
+
@Test
public void substituteNotPropertyId() {
compare("$foo.!", ImmutableMap.of("foo", false));
@@ -242,6 +328,11 @@ public class TemplateTest {
compare("\n$t.name.empty\n", ImmutableMap.of("t", Thread.currentThread()));
}
+ @Test
+ public void substituteUndefinedReference() {
+ expectException("$foo", ImmutableMap.of(), "Undefined reference $foo");
+ }
+
@Test
public void substituteMethodNoArgs() {
compare("<$c.size()>", ImmutableMap.of("c", ImmutableMap.of()));
@@ -259,8 +350,7 @@ public class TemplateTest {
@Test
public void substituteMethodOneNullArg() {
- // This should evaluate map.containsKey(map.get("absent")), which is map.containsKey(null).
- compare("<$map.containsKey($map.get(\"absent\"))>", ImmutableMap.of("map", ImmutableMap.of()));
+ compare("<$map.containsKey( null )>", ImmutableMap.of("map", ImmutableMap.of()));
}
@Test
@@ -268,6 +358,11 @@ public class TemplateTest {
compare("\n$s.indexOf(\"bar\", 2)\n", ImmutableMap.of("s", "barbarbar"));
}
+ @Test
+ public void substituteMethodExpressionArg() {
+ expectException("$sb.append(2 + 3) $sb", "Expected )");
+ }
+
@Test
public void substituteMethodSyntheticOverloads() {
// If we aren't careful, we'll see both the inherited `Set<K> keySet()` from Map
@@ -280,6 +375,15 @@ public class TemplateTest {
compare("$Integer.toHexString(23)", ImmutableMap.of("Integer", Integer.class));
}
+ @Test
+ public void substituteMethodNullLiteral() {
+ // Velocity recognizes the null literal, but only in this exact spot, as a method parameter.
+ // You can't say `#set($foo = null)` for example. Why not? Because.
+ compare(
+ "<$Objects.isNull(null) <$Objects.equals(null, null)>",
+ ImmutableMap.of("Objects", Objects.class));
+ }
+
@Test
public void substituteStaticMethodAsInstanceMethod() {
compare("$i.toHexString(23)", ImmutableMap.of("i", 0));
@@ -305,9 +409,48 @@ public class TemplateTest {
compare("$GetName.getName()", ImmutableMap.of("GetName", GetName.class));
}
+ @Test
+ public void substituteMethodOnNull() {
+ expectException(
+ "$foo.bar()",
+ Collections.singletonMap("foo", null),
+ "In $foo.bar(): $foo must not be null");
+ }
+
+ @Test
+ public void substituteMethodNonExistent() {
+ expectException(
+ "$i.nonExistent($i)",
+ ImmutableMap.of("i", 23),
+ "In $i.nonExistent($i): no method nonExistent in java.lang.Integer");
+ }
+
+ @Test
+ public void substituteMethodWrongArguments() {
+ expectException(
+ "$s.charAt()",
+ ImmutableMap.of("s", ""),
+ "In $s.charAt(): parameters for method charAt have wrong types: []");
+ expectException(
+ "$s.charAt('x')",
+ ImmutableMap.of("s", ""),
+ "In $s.charAt('x'): parameters for method charAt have wrong types: [x]");
+ }
+
+ @Test
+ public void substituteMethodAmbiguous() {
+ // Below, the null argument matches the (PrintStream) and the (PrintWriter) overloads.
+ // We don't test the method strings in the error because their exact format is unspecified.
+ expectException(
+ "$t.printStackTrace(null)",
+ ImmutableMap.of("t", new Throwable()),
+ "In $t.printStackTrace(null): ambiguous method invocation, could be one of:");
+ }
+
@Test
public void substituteIndexNoBraces() {
compare("<$map[\"x\"]>", ImmutableMap.of("map", ImmutableMap.of("x", "y")));
+ compare("<$map[ \"x\" ]>", ImmutableMap.of("map", ImmutableMap.of("x", "y")));
}
@Test
@@ -315,6 +458,22 @@ public class TemplateTest {
compare("<${map[\"x\"]}>", ImmutableMap.of("map", ImmutableMap.of("x", "y")));
}
+ @Test
+ public void substituteIndexNull() {
+ // Velocity allows null literals in method parameters but not indexes.
+ expectException(
+ "<$map[null]>",
+ ImmutableMap.of("map", ImmutableMap.of()),
+ "Identifier must be preceded by $");
+ }
+
+ @Test
+ public void substituteIndexExpression() {
+ // For no good reason, Velocity doesn't allow arbitrary expressions in indexes, so
+ // EscapeVelocity doesn't either.
+ expectException("<$map[2 + 3]>", "Expected ]");
+ }
+
// Velocity allows you to write $map.foo instead of $map["foo"].
@Test
public void substituteMapProperty() {
@@ -328,6 +487,82 @@ public class TemplateTest {
compare("<$map[2].name>", ImmutableMap.of("map", ImmutableMap.of(2, getClass())));
}
+ @Test
+ public void substituteNegativeIndex() {
+ // Negative index means n from the end, e.g. -1 is the last element of the list.
+ compare(
+ "$list[-1] $list[-2] $list[-3]",
+ ImmutableMap.of("list", ImmutableList.of("foo", "bar", "baz")));
+ }
+
+ @Test
+ public void substituteIndexOnNull() {
+ expectException(
+ "$foo[23]", Collections.singletonMap("foo", null), "In $foo[23]: $foo must not be null");
+ }
+
+ @Test
+ public void substituteListIndexNotInteger() {
+ expectException(
+ "$list[$list[0]]",
+ ImmutableMap.of("list", Collections.singletonList(null)),
+ "In $list[$list[0]]: list index is not an Integer: null");
+
+ // In V2, a string gets parsed into an integer here, resulting in NumberFormatException.
+ // We expect a VelocityException subclass in this test, so for now we skip this check.
+ assume().that(VERSION).isEqualTo(Version.V1);
+ expectException(
+ "$list['x']",
+ ImmutableMap.of("list", ImmutableList.of()),
+ "In $list['x']: list index is not an Integer: x");
+ }
+
+ @Test
+ public void substituteListIndexOutOfRange() {
+ expectException(
+ "$list[17]",
+ ImmutableMap.of("list", ImmutableList.of("foo")),
+ "In $list[17]: list index 17 is not valid for list of size 1");
+ expectException(
+ "$list[-2]",
+ ImmutableMap.of("list", ImmutableList.of("foo")),
+ "In $list[-2]: negative list index -2 counts from the end of the list, but the list size is"
+ + " only 1");
+ }
+
+ /**
+ * A class with a method that returns null. That means that {@code $x.null} and
+ * {@code $x.getNull()} both return null if {@code $x} is an instance of this class. If that null
+ * ends up being rendered in the output, it should be an error.
+ */
+ public static class NullHolder {
+ public Object getNull() {
+ return null;
+ }
+ }
+
+ /**
+ * Tests that it is an error if a null value gets rendered into the output. This is consistent
+ * with Velocity.
+ */
+ @Test
+ public void cantRenderNull() {
+ expectException("$x", Collections.singletonMap("x", null), "Null value for $x");
+ expectException("$x.null", ImmutableMap.of("x", new NullHolder()), "Null value for $x.null");
+ expectException("$x.null", ImmutableMap.of("x", ImmutableMap.of()), "Null value for $x.null");
+ expectException(
+ "$x.getNull()", ImmutableMap.of("x", new NullHolder()), "Null value for $x.getNull()");
+ expectException(
+ "$x['null']", ImmutableMap.of("x", ImmutableMap.of()), "Null value for $x['null']");
+ expectException(
+ "$x[\"null\"]", ImmutableMap.of("x", ImmutableMap.of()), "Null value for $x[\"null\"]");
+ }
+
+ @Test
+ public void canEvaluateNull() {
+ compare("#if ($foo == $foo) yes #end", Collections.singletonMap("foo", null));
+ }
+
@Test
public void variableNameCantStartWithNonAscii() {
compare("<$Éamonn>", ImmutableMap.<String, Object>of());
@@ -340,7 +575,7 @@ public class TemplateTest {
@Test
public void variableNameCharacters() {
- compare("<AZaz-foo_bar23>", ImmutableMap.of("AZaz-foo_bar23", "(P)"));
+ compare("<${AZaz-foo_bar23}>", ImmutableMap.of("AZaz-foo_bar23", "(P)"));
}
/**
@@ -370,6 +605,12 @@ public class TemplateTest {
compare("#set ($s = \"$x\") <$s>", ImmutableMap.of("x", "fred"));
compare("#set ($s = \"==$x$y\") <$s>", ImmutableMap.of("x", "fred", "y", "jim"));
compare("#set ($s = \"$x$y==\") <$s>", ImmutableMap.of("x", "fred", "y", "jim"));
+ compare("#set ($s = \"abc#if (true) yes #else no #{end}def\") $s");
+ compare("#set ($s = \"abc\ndef\nghi\") <$s>");
+ compare("#set ($s = \"<#double(17)>\") #macro(double $n) #set ($x = 2 * $n) $x #end $s");
+ Function<String, String> quote = s -> "«" + s + "»";
+ compare("<$quote.apply(\"#foreach ($a in $list)$a#end\")>",
+ ImmutableMap.of("quote", quote, "list", ImmutableList.of("foo", "bar", "baz")));
}
@Test
@@ -397,6 +638,40 @@ public class TemplateTest {
compare("foo #set ($x\n = 17)\nbar $x", ImmutableMap.<String, Object>of());
}
+ /**
+ * Tests the {@code #define} directive. This is mostly useless since macros do essentially the
+ * same thing and have parameters. We support it mainly because it is not much harder than
+ * documenting that we don't.
+ */
+ @Test
+ public void define() {
+ compare(
+ "#define ($hello) Hello, world #set ($x = $x + 1) $x #end $hello $hello",
+ ImmutableMap.of("x", 17));
+ compare(
+ "#define ($hello)\n Hello, world \n#set ($x = $x + 1) $x #end $hello $hello",
+ ImmutableMap.of("x", 17));
+ compare("#define ($hello) 23 #end #set ($hello = 17) $hello");
+ compare("#set ($hello = 17) #define ($hello) 23 skidoo #end $hello");
+ compare(
+ "Hello ${foo}!\n"
+ + "#define ($foo) darkness, my old friend #end\n"
+ + "Hello ${foo}!\n",
+ ImmutableMap.of("foo", "World"));
+ compare(
+ "#define ($recur) $x #set ($x = $x - 1) #if ($x > 0) $recur #end #end",
+ ImmutableMap.of("x", 5));
+ expectException(
+ "#define ($hello) $x #end #set ($x = 1) #set ($y = $hello.foo)",
+ "$hello comes from #define");
+ expectException(
+ "#define ($hello) $x #end #set ($x = 1) #set ($y = $hello.foo())",
+ "$hello comes from #define");
+ expectException(
+ "#define ($hello) $x #end #set ($x = 1) #set ($y = $hello[0])",
+ "$hello comes from #define");
+ }
+
@Test
public void expressions() {
compare("#set ($x = 1 + 1) $x");
@@ -406,6 +681,70 @@ public class TemplateTest {
compare("#set ($x = 22 - 7) $x");
compare("#set ($x = 22 / 7) $x");
compare("#set ($x = 22 % 7) $x");
+
+ compare("#set ($x = 'foo' + 'bar') $x");
+ compare("#set ($x = 23 + ' skidoo') $x");
+ compare("#set ($x = 'heaven ' + 17) $x");
+
+ // Check that we copy Velocity here: these null references will be replaced by their source
+ // text, for example "$bar" for the null $bar reference here.
+ compare("#set ($x = $bar + 'foo') $x", Collections.singletonMap("bar", null));
+
+ // In V2, if $bar is on the LHS of + it is as before, but on the RHS it is replaced by an empty
+ // string.
+ assume().that(VERSION).isEqualTo(Version.V1);
+
+ compare("#set ($x = 'foo' + $bar) $x", Collections.singletonMap("bar", null));
+
+ // This one results in "foo$bar + $bar" in both Velocity and EscapeVelocity. $bar + $bar is
+ // null and then 'foo' + null gets replaced by a representation of the source expression that
+ // produced the null.
+ compare("#set ($x = 'foo' + ($bar + $bar)) $x", Collections.singletonMap("bar", null));
+ }
+
+ @Test
+ public void divideByZeroIsNull() {
+ Map<String, Object> vars = new TreeMap<>();
+ vars.put("null", null);
+ Number[] values = {-1, 0, 23, Integer.MAX_VALUE};
+ for (Number value : values) {
+ vars.put("value", value);
+ compare("#set ($x = $value / 0) #if ($x == $null) null #else $x #end", vars);
+ compare("#set ($x = $value % 0) #if ($x == $null) null #else $x #end", vars);
+ }
+ }
+
+ @Test
+ public void arithmeticOperationsOnNullAreNull() {
+ String template =
+ Joiner.on('\n')
+ .join(
+ "#macro (nulltest $x) #if ($x == $null) is #else not #end null #end",
+ "#nulltest($null)",
+ "#nulltest('not null')",
+ "#set ($x = 1 + $null) #nulltest($x)",
+ "#set ($x = $null + $null) #nulltest($x)",
+ "#set ($x = $null - 1) #nulltest($x)",
+ "#set ($x = $null * $null) #nulltest($x)",
+ "#set ($x = $null / $null) #nulltest($x)",
+ "#set ($x = 3 / $null) #nulltest($x)",
+ "#set ($x = $null / 3) #nulltest($x)");
+ compare(template, Collections.singletonMap("null", null));
+ }
+
+ @Test
+ public void comparisonsOnNullFail() {
+ Map<String, Object> vars = new TreeMap<>();
+ vars.put("foo", null);
+ vars.put("bar", null);
+ expectException(
+ "#if ($foo < 1) null < 1 #end", vars, "Left operand $foo of < must not be null");
+ expectException(
+ "#if (1 < $foo) 1 < null #end", vars, "Right operand $foo of < must not be null");
+ expectException(
+ "#if ($foo < $bar) null < null #end", vars, "Left operand $foo of < must not be null");
+ expectException(
+ "#if ($foo >= $bar) null >= null #end", vars, "Left operand $foo of >= must not be null");
}
@Test
@@ -427,6 +766,10 @@ public class TemplateTest {
compare("#set ($x = false && true) $x");
compare("#set ($x = true && false) $x");
compare("#set ($x = true && true) $x");
+ compare("#set ($x = false and false) $x");
+ compare("#set ($x = false and true) $x");
+ compare("#set ($x = true and false) $x");
+ compare("#set ($x = true and true) $x");
}
@Test
@@ -435,12 +778,32 @@ public class TemplateTest {
compare("#set ($x = false || true) $x");
compare("#set ($x = true || false) $x");
compare("#set ($x = true || true) $x");
+ compare("#set ($x = false or false) $x");
+ compare("#set ($x = false or true) $x");
+ compare("#set ($x = true or false) $x");
+ compare("#set ($x = true or true) $x");
}
@Test
public void not() {
compare("#set ($x = !true) $x");
compare("#set ($x = !false) $x");
+ compare("#set ($x = not true) $x");
+ compare("#set ($x = not false) $x");
+ }
+
+ @Test
+ public void misspelledWordOperators() {
+ expectException(
+ "#if (no true) what #end", "Identifier must be preceded by $ or be true or false");
+ expectException(
+ "#if (nott true) what #end", "Identifier must be preceded by $ or be true or false");
+ expectException("#if (true oor false) what #end", "Expected 'or' but was 'oor");
+ expectException("#if (true andd false) what #end", "Expected 'and' but was 'andd");
+ expectException("#if (true annd false) what #end", "Expected 'and' but was 'annd");
+
+ // Neither Velocity nor EscapeVelocity has an xor operator.
+ expectException("#if (true xor false) what #end", "Expected )");
}
@Test
@@ -460,6 +823,40 @@ public class TemplateTest {
compare("#set ($x = " + Integer.MIN_VALUE + ") $x");
}
+ @Test
+ public void listLiterals() {
+ compare("#set ($list = []) $list");
+ compare("#set ($list = ['a', 'b', 'c']) $list");
+ compare("#set ($list = [ 1,2,3 ] ) $list");
+ compare("#foreach ($x in [$a, $b]) $x #end", ImmutableMap.of("a", 5, "b", 3));
+ compare("#set ($list = [ $null, $null ]) $list.size()", Collections.singletonMap("null", null));
+ // Like Velocity, we don't accept general expressions here.
+ expectException("#set ($list = [2 + 3])", "Expected ] at end of list literal");
+ // Test the toString():
+ expectException(
+ "$map[[1, 2, 3]]",
+ ImmutableMap.of("map", ImmutableMap.of()),
+ "Null value for $map[[1, 2, 3]]");
+ }
+
+ @Test
+ public void rangeLiterals() {
+ compare("#set ($range = [1..5]) $range");
+ compare("#set ($range = [5 .. 1]) $range");
+ compare("#foreach ($x in [-1 .. 5]) $x #end");
+ compare("#foreach ($x in [5..-1]) $x #end");
+ compare("#foreach ($x in [$a..$b]) $x #end", ImmutableMap.of("a", 3, "b", 5));
+ expectException("#set ($range = ['foo'..'bar'])", "Arithmetic is only available on integers");
+ // Like Velocity, we don't accept general expressions here.
+ expectException("#set ($list = [2 * 3 .. 10])", "Expected ] at end of list literal");
+ expectException("#set ($list = [10 .. 2 * 3])", "Expected ] at end of range literal");
+ // Test the toString():
+ expectException(
+ "$map[[1..3]]",
+ ImmutableMap.of("map", ImmutableMap.of()),
+ "Null value for $map[[1..3]]");
+ }
+
private static final String[] RELATIONS = {"==", "!=", "<", ">", "<=", ">="};
@Test
@@ -489,6 +886,10 @@ public class TemplateTest {
public void funkyEquals() {
compare("#set ($t = (123 == \"123\")) $t");
compare("#set ($f = (123 == \"1234\")) $f");
+
+ // In V2, two objects are equal if they implement CharSequence and their characters are the
+ // same.
+ assume().that(VERSION).isEqualTo(Version.V1);
compare("#set ($x = ($sb1 == $sb2)) $x", ImmutableMap.of(
"sb1", (Object) new StringBuilder("123"),
"sb2", (Object) new StringBuilder("123")));
@@ -503,6 +904,7 @@ public class TemplateTest {
compare("x#if (true) y #end\nz");
compare("x#if (true)\ny #end\nz");
compare("x#if (true) y #end\nz");
+ compare("x#if (true) y #end\n\nz");
compare("$x #if (true) y #end $x ", ImmutableMap.of("x", "!"));
}
@@ -534,6 +936,7 @@ public class TemplateTest {
@Test
public void ifFalseWithElseIfTrue() {
compare("x#if (false) a #elseif (true) b #else c #end z");
+ compare("x#if (false)\na\n#elseif (true)\nb\n#else\nc\n#end\nz");
}
@Test
@@ -548,18 +951,61 @@ public class TemplateTest {
@Test
public void ifUndefined() {
compare("#if ($undefined) really? #else indeed #end");
+ compare("#if ($false || $undefined) nope #else yes #end", ImmutableMap.of("false", false));
+ compare("#if ($true && $undefined) nope #else yes #end", ImmutableMap.of("true", true));
+ compare("#if (!$undefined) yes #else nope #end");
+
+ // Only plain references get this special treatment and only when being evaluated for truth.
+ expectException("#if ($foo.bar) oops #end", "Undefined reference $foo");
+ expectException("#if ($foo.bar()) oops #end", "Undefined reference $foo");
+ expectException("#if ($foo[0]) oops #end", "Undefined reference $foo");
+ expectException(
+ "#if ($list[$foo]) oops #end",
+ ImmutableMap.of("list", ImmutableList.of("foo", "bar", "baz")),
+ "Undefined reference $foo");
+ expectException(
+ "#if ($undefined1 == $undefined2) yes #else nope #end", "Undefined reference $undefined1");
+
+ // The special treatment is only in #if, not when evaluating Boolean expressions in general.
+ expectException("#set ($foo = !$undefined) $foo", "Undefined reference $undefined");
+ expectException(
+ "#set ($foo = $false || $undefined) $foo",
+ ImmutableMap.of("false", false),
+ "Undefined reference $undefined");
}
@Test
public void forEach() {
compare("x#foreach ($x in $c) <$x> #end y",
ImmutableMap.of("c", ImmutableList.of()));
+ compare("x#foreach ($x in $c) <$x> #end\ny",
+ ImmutableMap.of("c", ImmutableList.of()));
+ compare("x#foreach ($x in $c) <$x> #end\n\ny",
+ ImmutableMap.of("c", ImmutableList.of()));
compare("x#foreach ($x in $c) <$x> #end y",
ImmutableMap.of("c", ImmutableList.of("foo", "bar", "baz")));
compare("x#foreach ($x in $c) <$x> #end y",
ImmutableMap.of("c", new String[] {"foo", "bar", "baz"}));
compare("x#foreach ($x in $c) <$x> #end y",
ImmutableMap.of("c", ImmutableMap.of("foo", "bar", "baz", "buh")));
+ compare("x#foreach (${x} in $c) <$x> #end y",
+ ImmutableMap.of("c", ImmutableMap.of("foo", "bar", "baz", "buh")));
+ compare("x#foreach ($!{x} in $c) <$x> #end y",
+ ImmutableMap.of("c", ImmutableMap.of("foo", "bar", "baz", "buh")));
+ }
+
+ @Test
+ public void forEachBad() {
+ expectException("#foreach (x in $c) <$x> #end", "Expected variable beginning with '$'");
+ expectException("#foreach ($x.foo in $c) <$x> #end", "Expected simple variable");
+ expectException("#foreach ($ in $c) #end", "Expected simple variable");
+ }
+
+ @Test
+ public void forEachNull() {
+ // Bizarrely, Velocity allows you to iterate on null, with no effect.
+ Map<String, Object> vars = Collections.singletonMap("null", null);
+ compare("#foreach ($x in $null) $x #end", vars);
}
@Test
@@ -570,6 +1016,14 @@ public class TemplateTest {
ImmutableMap.of("c", ImmutableList.of("foo", "bar", "baz")));
}
+ @Test
+ public void forEachFirstLast() {
+ compare("x#foreach ($x in $c) <#if ($foreach.first)?#end$x#if ($foreach.last)!#end#end",
+ ImmutableMap.of("c", ImmutableList.of()));
+ compare("x#foreach ($x in $c) <#if ($foreach.first)?#end$x#if ($foreach.last)!#end#end",
+ ImmutableMap.of("c", ImmutableList.of("foo", "bar", "baz")));
+ }
+
@Test
public void nestedForEach() {
String template =
@@ -609,16 +1063,61 @@ public class TemplateTest {
compare(template, ImmutableMap.of("list", ImmutableList.of("blim", "blam", "blum")));
}
+ @Test
+ public void forEachCount() {
+ String template =
+ "#foreach ($x in $list)"
+ + "[$foreach.count]"
+ + "#foreach ($y in $list)"
+ + "($foreach.count)==$x.$y=="
+ + "#end"
+ + "#end";
+ compare(template, ImmutableMap.of("list", ImmutableList.of("blim", "blam", "blum")));
+ }
+
+ @Test
+ public void forEachForEach() {
+ // There's no reason to do this, but if you do you get {} for $foreach.
+ compare("#foreach ($x in [1..10]) [$foreach] #end");
+ }
+
+ @Test
+ public void breakDirective() {
+ compare("#foreach ($x in [1..10]) $x #if ($x == 5) #break #end #end");
+ compare("#foreach ($x in [1..10]) $x #if ($x == 5)\n#break\n#end #end");
+ compare("#foreach ($x in [1..10]) $x #if ($x == 5)\n#break ($foreach)\n#end #end");
+ compare("#foreach ($x in [1..10]) $x #if ($x == 5)\n#break\n($foreach)\nignored#end #end");
+ compare("#foreach ($x in [1..10]) $x #set ($f = $foreach) #break($f) #end");
+ compare("foo bar #break baz"); // should render as "foo bar ".
+ expectException(
+ "#foreach ($x in [1..10]) #break($null) #end",
+ Collections.singletonMap("null", null),
+ "Argument to #break is not a supported scope: $null");
+ expectException("foo bar #break($foreach) baz", "Undefined reference $foreach");
+ expectException("#set ($x = 17) #break($x)", "Argument to #break is not a supported scope: $x");
+ expectException("#foreach ($x in [1..10]) $x #break($foreach #end", "Expected )");
+ }
+
@Test
public void setSpacing() {
// The spacing in the output from #set is eccentric.
+ // If the #set is preceded by a reference or a comment or a directive (for example another #set)
+ // with only horizontal space intervening, that space is deleted. But if there are newlines,
+ // nothing is deleted. But, a newline immediately after a directive or comment is already
+ // deleted, and if only horizontal space remains before the #set, it is deleted.
+ compare(" #set ($x = 1)"); // preceding space is deleted
compare("x#set ($x = 0)x");
compare("x #set ($x = 0)x");
compare("x #set ($x = 0) x");
compare("$x#set ($x = 0)x", ImmutableMap.of("x", "!"));
+ compare("x#set ($foo = 'bar')\n#set ($baz = 'buh')\n!");
+ compare("x#if (1 + 1 == 2) ok #else ? #end\n#set ($foo = 'bar')\ny");
+ compare("x#if (1 + 1 == 2) ok #else ? #end #set ($foo = 'bar')\ny");
- // Velocity WTF: the #set eats the space after $x and other references, so the output is <!x>.
compare("$x #set ($x = 0)x", ImmutableMap.of("x", "!"));
+ compare("$x\n#set ($x = 0)x", ImmutableMap.of("x", "!"));
+ compare("#set($x = 0)\n#set($y = 1)\n<$x$y>");
+ compare("#set($x = 0)\n #set($y = 1)\n<$x$y>");
compare("$x.length() #set ($x = 0)x", ImmutableMap.of("x", "!"));
compare("$x.empty #set ($x = 0)x", ImmutableMap.of("x", "!"));
compare("$x[0] #set ($x = 0)x", ImmutableMap.of("x", ImmutableList.of("!")));
@@ -629,8 +1128,51 @@ public class TemplateTest {
compare("x ## comment\n #set($x = 0) y");
compare("x #* comment *# #set($x = 0) y");
+ compare("$list.size()\n#set ($foo = 'bar')\n!", ImmutableMap.of("list", ImmutableList.of()));
+ compare("$list[0]\n #set ($foo = 'bar')\n!", ImmutableMap.of("list", ImmutableList.of("x")));
+
+ compare(" #set ($x = 3)\n");
+ compare("\n\n#set ($x = 3)\n");
+ compare("\n\n #set ($x = 3)\n");
+ compare(
+ " #foreach ($i in [1..3])\n #set ($j = \"!$i!\")\n #set ($k = $i + 1)\n $j$k\n #end");
+
+ compare(
+ "#set ($foo = 17)\n" //
+ + "#set ($bar = 23)\n"
+ + "\n"
+ + "#set ($baz = 5)\n"
+ + "hello");
+
+ compare(
+ "## comment\n" //
+ + "\n"
+ + "\n"
+ + "#set ($foo = 17)\n"
+ + "hello");
+
+ compare(
+ "## comment\n" //
+ + "\n"
+ + " #set ($foo = 17)\n"
+ + "hello");
+
+ compare(
+ "#macro (m)\n\n\n\n" //
+ + "#set ($foo = 17)\n"
+ + "hello\n"
+ + "#end\n"
+ + "#m()\n");
+
+ compare(
+ "#macro (m)\n" //
+ + " #set ($foo = 17)\n"
+ + "hello\n"
+ + "#end\n"
+ + "#m()\n");
}
+
@Test
public void simpleMacro() {
String template =
@@ -666,6 +1208,20 @@ public class TemplateTest {
compare(template, ImmutableMap.of("x", "tiddly"));
}
+ @Test
+ public void recursiveMacro() {
+ String template =
+ "#macro (m $s)\n"
+ + "$s\n"
+ + " #if (! $s.empty)\n"
+ + " #set ($s = $s.substring(1))\n"
+ + " #m($s)\n"
+ + " #end\n"
+ + "#end\n"
+ + "#m('foobar')\n";
+ compare(template);
+ }
+
/**
* Tests defining a macro inside a conditional. This proves that macros are not evaluated in the
* main control flow, but rather are extracted at parse time. It also tests what happens if there
@@ -759,6 +1315,8 @@ public class TemplateTest {
@Test
public void callByMacro() {
+ // In V2, macro arguments are call-by-value rather than call-by-name.
+ assume().that(VERSION).isEqualTo(Version.V1);
// Since #callByMacro1 never references its argument, $x.add("t") is never evaluated during it.
// Since #callByMacro2 references its argument twice, $x.add("t") is evaluated twice during it.
String template =
@@ -802,6 +1360,9 @@ public class TemplateTest {
@Test
public void nameCaptureSwap() {
+ // In V2, macro arguments are call-by-value rather than call-by-name.
+ assume().that(VERSION).isEqualTo(Version.V1);
+
// Here, the arguments $a and $b are variables rather than literals, which means that their
// values change when we set those variables. #set($tmp = $a) changes the meaning of $b since
// $b is the name $tmp. So #set($a = $b) shadows parameter $a with the value of $tmp, which we
@@ -824,13 +1385,13 @@ public class TemplateTest {
@Test
public void badBraceReference() {
String template = "line 1\nline 2\nbar${foo.!}baz";
- expectParseException(template, "Expected }, on line 3, at text starting: .!}baz");
+ expectException(template, "Expected }, on line 3, at text starting: .!}baz");
}
@Test
public void undefinedMacro() {
String template = "#oops()";
- expectParseException(
+ expectException(
template,
"#oops is neither a standard directive nor a macro that has been defined");
}
@@ -840,13 +1401,78 @@ public class TemplateTest {
String template =
"#macro (twoArgs $a $b) $a $b #end\n"
+ "#twoArgs(23)\n";
- expectParseException(template, "Wrong number of arguments to #twoArgs: expected 2, got 1");
+ expectException(template, "Wrong number of arguments to #twoArgs: expected 2, got 1");
+ }
+
+ @Test
+ public void macroWithBody() {
+ // The #if ($bodyContent) is needed because Velocity treats $bodyContent as undefined if the
+ // macro is invoked as #withBody rather than #@withBody. The documented examples use
+ // $!bodyContent but that doesn't work if there is no body and Velocity is in strict ref mode.
+ String template =
+ "#macro(withBody $x)\n"
+ + "$x\n#if ($bodyContent) $bodyContent #end\n"
+ + "#end\n"
+ + "#withBody('foo')\n"
+ + "#@withBody('bar')\n"
+ + "#if (0 == 1) what #else yes #end\n"
+ + "#end ## end of #@withBody";
+ compare(template);
+ }
+
+ @Test
+ public void nestedMacrosWithBodies() {
+ String template =
+ "#macro(outer $x)\n"
+ + "$x $!bodyContent $x\n"
+ + "#end\n"
+ + "#macro(inner $x)\n"
+ + "[$x] $!bodyContent [$x]\n"
+ + "#end\n"
+ + "#@outer('foo')\n"
+ + "before inner\n"
+ + "#@inner('bar')\n"
+ + "in inner\n"
+ + "#end ## inner\n"
+ + "after inner\n"
+ + "#end ## outer\n";
+ compare(template);
+ }
+
+ @Test
+ public void bodyContentTwice() {
+ String template =
+ "#macro(one)\n"
+ + "[$bodyContent]\n"
+ + "#end\n"
+ + "#macro(two)\n"
+ + "#set($bodyContentCopy = \"$bodyContent\")\n"
+ + "<#@one()$bodyContentCopy#end>\n"
+ + "#end\n"
+ + "#@two()foo#end\n";
+ // Velocity doesn't handle this well. It shouldn't be necessary to make $bodyContentCopy; we
+ // should just be able to use $bodyContent. But that gets an exception: "Reference $bodyContent
+ // evaluated to object org.apache.velocity.runtime.directive.Block$Reference whose toString()
+ // method returned null". By evaluating it into $bodyContentCopy we avoid whatever confusion
+ // that was. This is probably related to
+ // https://issues.apache.org/jira/projects/VELOCITY/issues/VELOCITY-940.
+ compare(template);
+ }
+
+ @Test
+ public void notMacroCall() {
+ // In V2, you don't need () after a parameterless macro call.
+ assume().that(VERSION).isEqualTo(Version.V1);
+ compare("#@ foo");
+ compare("#@foo no parens");
}
@Test
public void unclosedBlockQuote() {
+ // This is not an error in V2. https://issues.apache.org/jira/browse/VELOCITY-962
+ assume().that(VERSION).isEqualTo(Version.V1);
String template = "foo\nbar #[[\nblah\nblah";
- expectParseException(template, "Unterminated #[[ - did not see matching ]]#, on line 2");
+ expectException(template, "Unterminated #[[ - did not see matching ]]#, on line 2");
}
@Test
@@ -854,56 +1480,89 @@ public class TemplateTest {
compare("foo\nbar #*\nblah\nblah");
}
- /**
- * A Velocity ResourceLoader that looks resources up in a map. This allows us to test directives
- * that read "resources", for example {@code #parse}, without needing to make separate files to
- * put them in.
- */
- private static final class MapResourceLoader extends ResourceLoader {
- private final ImmutableMap<String, String> resourceMap;
+ @Test
+ public void evaluate() {
+ compare("#evaluate('foo $x bar')", ImmutableMap.of("x", "baz"));
+ compare("#evaluate('foo #set ($x = 17) $x bar')");
+ compare("#evaluate($x) $y", ImmutableMap.of("x", "#set ($y = 'foo')"));
+ compare("#set($nested = '#set ($y = \"foo\")')\n"
+ + "#evaluate('#evaluate ($nested)')\n"
+ + "$y");
+ compare("#evaluate ('$x')\nnext line", ImmutableMap.of("x", "foo"));
+ compare("#evaluate($null)", Collections.singletonMap("null", null));
+ expectException("#evaluate(23)", "Argument to #evaluate must be a string: 23");
+ }
- MapResourceLoader(ImmutableMap<String, String> resourceMap) {
- this.resourceMap = resourceMap;
- }
+ @Test
+ public void nullReference() throws IOException {
+ Map<String, Object> vars = Collections.singletonMap("foo", null);
+ expectException("==$foo==", vars, "Null value for $foo");
+ compare("==$!foo==", vars);
+ }
- @Override
- public void init(ExtendedProperties configuration) {
- }
+ @Test
+ public void nullMethodCall() throws IOException {
+ Map<String, Object> vars = ImmutableMap.of("map", ImmutableMap.of());
+ expectException("==$map.get(23)==", vars, "Null value for $map.get(23)");
+ compare("==$!map.get(23)==", vars);
+ }
- @Override
- public InputStream getResourceStream(String source) {
- String resource = resourceMap.get(source);
- if (resource == null) {
- throw new ResourceNotFoundException(source);
- }
- return new ByteArrayInputStream(resource.getBytes(StandardCharsets.ISO_8859_1));
- }
+ @Test
+ public void nullIndex() throws IOException {
+ Map<String, Object> vars = ImmutableMap.of("map", ImmutableMap.of());
+ expectException("==$map[23]==", vars, "Null value for $map[23]");
+ compare("==$!map[23]==", vars);
+ }
- @Override
- public boolean isSourceModified(Resource resource) {
- return false;
- }
+ @Test
+ public void ifNull() throws IOException {
+ // Null references in #if mean false.
+ Map<String, Object> vars = Collections.singletonMap("nullRef", null);
+ compare("#if ($nullRef) oops #end", vars);
+ }
- @Override
- public long getLastModified(Resource resource) {
- return 0;
- }
- };
+ @Test
+ public void nullProperty() throws IOException {
+ // We use a LinkedList with a null element so that list.getFirst() will return null. Then
+ // $list.first is a null reference.
+ @SuppressWarnings("JdkObsolete")
+ LinkedList<String> list = new LinkedList<>();
+ list.add(null);
+ Map<String, Object> vars = ImmutableMap.of("list", list);
+ expectException("==$list.first==", vars, "Null value for $list.first");
+ compare("==$!list.first==", vars);
+ }
+
+ @Test
+ public void silentRefInDirective() throws IOException {
+ Map<String, Object> vars = new TreeMap<>();
+ vars.put("null", null);
+ compare("#if ($!null == '') yes #end", vars);
+ }
+
+ @Test
+ public void silentRefInString() throws IOException {
+ Map<String, Object> vars = Collections.singletonMap("null", null);
+ compare("#set ($nuller = \"$!{null}er\") $nuller", vars);
+ }
private String renderWithResources(
String templateResourceName,
ImmutableMap<String, String> resourceMap,
- ImmutableMap<String, String> vars) {
- MapResourceLoader mapResourceLoader = new MapResourceLoader(resourceMap);
- RuntimeInstance runtimeInstance = newVelocityRuntimeInstance();
- runtimeInstance.setProperty("resource.loader", "map");
- runtimeInstance.setProperty("map.resource.loader.instance", mapResourceLoader);
- runtimeInstance.init();
- org.apache.velocity.Template velocityTemplate =
- runtimeInstance.getTemplate(templateResourceName);
+ Map<String, String> vars) {
+ String name = testName.getMethodName();
+ VelocityEngine engine = newVelocityEngine();
+ engine.setProperty("resource.loader", "string");
+ engine.setProperty("string.resource.loader.class", StringResourceLoader.class.getName());
+ engine.setProperty("string.resource.loader.repository.name", name);
+ engine.init();
+ StringResourceRepository repo = StringResourceLoader.getRepository(name);
+ resourceMap.forEach(repo::putStringResource);
StringWriter velocityWriter = new StringWriter();
VelocityContext velocityContext = new VelocityContext(new TreeMap<>(vars));
- velocityTemplate.merge(velocityContext, velocityWriter);
+ boolean rendered =
+ engine.mergeTemplate(templateResourceName, "UTF-8", velocityContext, velocityWriter);
+ assertThat(rendered).isTrue();
return velocityWriter.toString();
}
@@ -911,11 +1570,14 @@ public class TemplateTest {
public void parseDirective() throws IOException {
// If outer.vm does #parse("nested.vm"), then we should be able to #set a variable in
// nested.vm and use it in outer.vm, and we should be able to define a #macro in nested.vm
- // and call it in outer.vm.
+ // and call it in outer.vm. However, if a macro is defined in outer.vm then a later definition
+ // in nested.vm will be ignored.
ImmutableMap<String, String> resources = ImmutableMap.of(
"outer.vm",
"first line\n"
+ + "#macro (alreadyDefined) already defined #end\n"
+ "#parse (\"nested.vm\")\n"
+ + "#alreadyDefined()\n"
+ "<#decorate (\"left\" \"right\")>\n"
+ "$baz skidoo\n"
+ "last line\n",
@@ -923,6 +1585,8 @@ public class TemplateTest {
"nested template first line\n"
+ "[#if ($foo == $bar) equal #else not equal #end]\n"
+ "#macro (decorate $a $b) < $a | $b > #end\n"
+ + "#macro (alreadyDefined) not redefined #end\n"
+ + "#alreadyDefined()\n"
+ "#set ($baz = 23)\n"
+ "nested template last line\n");
@@ -950,5 +1614,92 @@ public class TemplateTest {
assertThat(e).hasMessageThat().isEqualTo(
"In expression on line 2 of nested.vm: Undefined reference $bar");
}
+
+ // If we evaluate just nested.vm, we should see it use its own definition of #alreadyDefined,
+ // which went unused above because the one in outer.vm took precedence.
+ compare(resources.get("nested.vm"), ImmutableMap.of("foo", "foovalue", "bar", "barvalue"));
+ }
+
+ @Test
+ public void parseDirectiveWithExpression() throws IOException {
+ // This tests evaluating the same template with different variables, where the variables
+ // determine which other template should be included by #parse. We should notably see that the
+ // macros defined differ on each evaluation, since the included templates have different
+ // definitions for the #decorate macro.
+ ImmutableMap<String, String> resources = ImmutableMap.of(
+ "outer.vm",
+ "first line\n"
+ + "#parse (\"${nested}.vm\")\n"
+ + "<#decorate (\"left\" \"right\")>\n"
+ + "$baz skidoo\n"
+ + "last line\n",
+ "nested.vm",
+ "nested template first line\n"
+ + "[#if ($foo == $bar) equal #else not equal #end]\n"
+ + "#macro (decorate $a $b) < $a | $b > #end\n"
+ + "#set ($baz = 23)\n"
+ + "nested template last line\n",
+ "othernested.vm",
+ "#macro (decorate $a $b) [ $a | $b ] #end\n"
+ + "#set ($baz = 17)\n"
+ + "#break\n" // this breaks from the #parse of othernested.vm
+ + "not included");
+ ImmutableMap<String, String> vars = ImmutableMap.of(
+ "foo", "foovalue", "bar", "barvalue", "nested", "nested");
+
+ String velocityResult = renderWithResources("outer.vm", resources, vars);
+
+ Set<String> openedResources = new TreeSet<>();
+ Template.ResourceOpener resourceOpener = resourceName -> {
+ assertThat(resources).containsKey(resourceName);
+ assertThat(openedResources).doesNotContain(resourceName);
+ String resource = resources.get(resourceName);
+ openedResources.add(resourceName);
+ return new StringReader(resource);
+ };
+ Template template = Template.parseFrom("outer.vm", resourceOpener);
+
+ String result = template.evaluate(vars);
+ assertThat(result).isEqualTo(velocityResult);
+
+ Map<String, String> newVars = new LinkedHashMap<>(vars);
+ newVars.put("nested", "othernested");
+ String newVelocityResult = renderWithResources("outer.vm", resources, newVars);
+ String newResult = template.evaluate(newVars);
+ assertThat(newResult).isEqualTo(newVelocityResult);
+
+ // This should not read othernested.vm again (the second assertion in resourceOpener above).
+ String newResultAgain = template.evaluate(newVars);
+ assertThat(newResultAgain).isEqualTo(newResult);
+ }
+
+ @Test
+ public void parseDirectiveWithParseException() throws IOException {
+ ImmutableMap<String, String> resources =
+ ImmutableMap.of("outer.vm", "#parse('bad.vm')", "bad.vm", "#end");
+ Template.ResourceOpener resourceOpener =
+ resourceName -> new StringReader(resources.get(resourceName));
+ Template template = Template.parseFrom("outer.vm", resourceOpener);
+ EvaluationException e =
+ assertThrows(EvaluationException.class, () -> template.evaluate(ImmutableMap.of()));
+ assertThat(e).hasCauseThat().isInstanceOf(ParseException.class);
+ }
+
+ @Test
+ public void parseDirectiveWithIoException() throws IOException {
+ ImmutableMap<String, String> resources = ImmutableMap.of("outer.vm", "#parse('bad.vm')");
+ Template.ResourceOpener resourceOpener =
+ resourceName -> {
+ String resource = resources.get(resourceName);
+ if (resource == null) {
+ throw new FileNotFoundException(resourceName);
+ }
+ return new StringReader(resource);
+ };
+ Template template = Template.parseFrom("outer.vm", resourceOpener);
+ EvaluationException e =
+ assertThrows(EvaluationException.class, () -> template.evaluate(ImmutableMap.of()));
+ assertThat(e).hasCauseThat().isInstanceOf(FileNotFoundException.class);
+ assertThat(e).hasCauseThat().hasMessageThat().isEqualTo("bad.vm");
}
}