New Upstream Release - golang-github-zclconf-go-cty
Ready changes
Summary
Merged new upstream version: 1.13.1 (was: 1.13.0).
Diff
diff --git a/CHANGELOG.md b/CHANGELOG.md
index a641e4e..5b2e242 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,28 @@
+# 1.13.1 (March 16, 2023)
+
+* `function`: If a function parameter that doesn't declare `AllowDynamicType: true` recieves a `cty.DynamicVal`, the function system would previously just skip calling the function's `Type` callback and treat the result type as unknown. However, the `Call` method was then still calling a function's `Impl` callback anyway, which violated the usual contract that `Type` acts as a guard for `Impl` so `Impl` doesn't have to repeat type-checking already done in `Type`: it's only valid to call `Impl` if `Type` was previosly called _and_ it succeeded.
+
+ The function system will now skip calling `Impl` if it skips calling `Type`, immediately returning `cty.DynamicVal` in that case. Individual functions can opt out of this behavior by marking one or more of their parameters as `AllowDynamicType: true` and then handling that situation manually inside the `Type` and `Impl` callbacks.
+
+ As a result of this problem, some of the `function/stdlib` functions were not correctly handling `cty.DynamicVal` arguments after being extended to support refinements in the v1.13.0 release, causing unexpected errors or panics when calling them. Those functions are fixed indirectly by this change, since their callbacks will no longer run at all in those cases, as was true before they were extended to support refinements.
+
+# 1.13.0 (February 23, 2023)
+
+## Upgrade Notes
+
+This release introduces a new concept called [Refinements](./docs/refinements.md), which allow `cty` to constrain the range of an unknown value beyond just a type constraint and then make deductions about validity or result range based on those refinements.
+
+These changes are consistent with [the backward-compatibility policy](COMPATIBILITY.md) but you may see some changed results in your unit tests of operations involving unknown values. If the new results don't seem like valid refinements of what was previously being returned in the v1.12 series, please open an issue to discuss that.
+
+If the new results have a range that is a valid subset of the old results then that is expected behavior and you should update your tests as part of upgrading.
+
+## Other changes in this release
+
+* Refinements: `cty` will can track a refined range for some unknown values and will take those into account when evaluating certain operations, thereby allowing a "more known" result than before. ([#153](https://github.com/zclconf/go-cty/pull/153))
+* `function/stdlib`: The `FormatDate` and `TimeAdd` functions in previous releases were accidentally more liberal than intended in their interpretation of timestamp strings documented as requiring RFC3339. ([#152](https://github.com/zclconf/go-cty/pull/152))
+
+ Those functions are now corrected to use a stricter RFC3339 parser, meaning that they will now reject some inputs that were previously accepted but were not valid per the RFC3339 syntax rules. The documentation for these functions already specified that RFC3339 syntax was required and so this is a fix to a defect rather than a breaking change, but calling applications which embed these functions may wish to pass on an upgrade note about this behavior difference in their own releaase notes after upgrading.
+
# 1.12.1 (November 8, 2022)
* `convert`: Will now produce correct type constraints when the input value is an empty collection and the target element type has optional attributes. In this case the conversion process must remove the optional attribute annotations because those are only for type conversion purposes and have no meaning when used in the type constraint for an empty collection. ([#143](https://github.com/zclconf/go-cty/pull/143))
diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md
new file mode 100644
index 0000000..02cbe65
--- /dev/null
+++ b/COMPATIBILITY.md
@@ -0,0 +1,88 @@
+# `cty` backward-compatibility policy
+
+This library includes a number of behaviors that aim to support "best effort"
+partial evaluation in the presence of wholly- or partially-unknown inputs.
+Over time we've improved the accuracy of those analyses, but doing so changes
+the specific results returned by certain operations.
+
+This document aims to describe what sorts of changes are allowed in new minor
+releases and how those changes might affect the behavior of dependents after
+upgrading.
+
+Where possible we'll avoid making changes like these in _patch_ releases, which
+focus instead only on correcting incorrect behavior. An exception would be if
+a minor release introduced an incorrect behavior and then a patch release
+repaired it to either restore the previous correct behavior or implement a new
+compromise correct behavior.
+
+## Unknown Values can become "more known"
+
+The most significant policy is that any operation that was previously returning
+an unknown value may return either a known value or a _more refined_ unknown
+value in later releases, as long as the new result is a subset of the range
+of the previous result.
+
+When using only the _operation methods_ and functionality derived from them,
+`cty` will typically handle these deductions automatically and return the most
+specific result it is able to. In those cases we expect that these changes will
+be seen as an improvement for end-users, and not require significant changes
+to calling applications to pass on those benefits.
+
+When working with _integration methods_ (those which return results using
+"normal" Go types rather than `cty.Value`) these changes can be more sigificant,
+because applications can therefore observe the differences more readily.
+For example, if an unknown value is replaced with a known value of the same
+type then `Value.IsKnown` will begin returning `true` where it previously
+returned `false`. Applications should be designed to avoid depending on
+specific implementation details like these and instead aim to be more general
+to handle both known and unknown values.
+
+A specific sensitive area for compatibility is the `Value.RawEquals` method,
+which is sensitive to all of the possible variations in values. Applications
+should not use this method for normal application code to avoid exposing
+implementation details to end-users, but might use it to assert exact expected
+results in unit tests. Such test cases may begin failing after upgrading, and
+application developers should carefully consider whether the new results conform
+to these rules and update the tests to match as part of their upgrade if so. If
+the changed result seems _not_ to conform to these rules then that might be a
+bug; please report it!
+
+## Error situations may begin succeeding
+
+Over time the valid inputs or other constraints on functionality might be
+loosened to support new capabilities. Any operation or function that returned
+an error in a previous release can begin succeeding with any valid result in
+a new release.
+
+## Error message text might change
+
+This library aims to generate good, actionable error messages for user-facing
+problems and to give sufficient information to a calling application to generate
+its own high-quality error messages in situations where `cty` is not directly
+"talking to" an end-user.
+
+This means that in later releases the exact text of error messages in certain
+situations may change, typically to add additional context or increase
+precision.
+
+If a function is documented as returning a particular error type in a certain
+situation then that should be preserved in future releases, but if there is
+no explicit documentation then calling applications should not depend on the
+dynamic type of any `error` result, or should at least do so cautiously with
+a fallback to a general error handler.
+
+## Passing on changes to Go standard library
+
+Some parts of `cty` are wrappers around functionality implemented in the Go
+standard library. If the underlying packages change in newer versions of Go
+then we may or may not pass on the change through the `cty` API, depending on
+the circumstances.
+
+A specific notable example is Unicode support: this library depends on various
+Unicode algorithms and data tables indirectly through its dependencies,
+including some in the Go standard library, and so its exact treatment of strings
+is likely to vary between releases as the Unicode standard grows. We aim to
+follow the version of Unicode supported in the latest version of the Go standard
+library, although we may lag behind slightly after new Go releases due to the
+need to update other libraries that implement other parts of the Unicode
+specifications.
diff --git a/README.md b/README.md
index d0b48a8..4ab44e1 100644
--- a/README.md
+++ b/README.md
@@ -42,6 +42,9 @@ For more details, see the following documentation:
* [Conversion to and from native Go values](./docs/gocty.md)
* [JSON serialization](./docs/json.md)
* [`cty` Functions system](./docs/functions.md)
+* [Compatibility Policy for future Minor Releases](./COMPATIBILITY.md): please
+ review this before using `cty` in your application to avoid depending on
+ implementation details that may change.
---
diff --git a/cty/convert/conversion.go b/cty/convert/conversion.go
index 541b9a4..bc79df8 100644
--- a/cty/convert/conversion.go
+++ b/cty/convert/conversion.go
@@ -43,7 +43,7 @@ func getConversion(in cty.Type, out cty.Type, unsafe bool) conversion {
out = out.WithoutOptionalAttributesDeep()
if !isKnown {
- return cty.UnknownVal(dynamicReplace(in.Type(), out)), nil
+ return prepareUnknownResult(in.Range(), dynamicReplace(in.Type(), out)), nil
}
if isNull {
@@ -199,3 +199,64 @@ func retConversion(conv conversion) Conversion {
return conv(in, cty.Path(nil))
}
}
+
+// prepareUnknownResult can apply value refinements to a returned unknown value
+// in certain cases where characteristics of the source value or type can
+// transfer into range constraints on the result value.
+func prepareUnknownResult(sourceRange cty.ValueRange, targetTy cty.Type) cty.Value {
+ sourceTy := sourceRange.TypeConstraint()
+
+ ret := cty.UnknownVal(targetTy)
+ if sourceRange.DefinitelyNotNull() {
+ ret = ret.RefineNotNull()
+ }
+
+ switch {
+ case sourceTy.IsObjectType() && targetTy.IsMapType():
+ // A map built from an object type always has the same number of
+ // elements as the source type has attributes.
+ return ret.Refine().CollectionLength(len(sourceTy.AttributeTypes())).NewValue()
+ case sourceTy.IsTupleType() && targetTy.IsListType():
+ // A list built from a typle type always has the same number of
+ // elements as the source type has elements.
+ return ret.Refine().CollectionLength(sourceTy.Length()).NewValue()
+ case sourceTy.IsTupleType() && targetTy.IsSetType():
+ // When building a set from a tuple type we can't exactly constrain
+ // the length because some elements might coalesce, but we can
+ // guarantee an upper limit. We can also guarantee at least one
+ // element if the tuple isn't empty.
+ switch l := sourceTy.Length(); l {
+ case 0, 1:
+ return ret.Refine().CollectionLength(l).NewValue()
+ default:
+ return ret.Refine().
+ CollectionLengthLowerBound(1).
+ CollectionLengthUpperBound(sourceTy.Length()).
+ NewValue()
+ }
+ case sourceTy.IsCollectionType() && targetTy.IsCollectionType():
+ // NOTE: We only reach this function if there is an available
+ // conversion between the source and target type, so we don't
+ // need to repeat element type compatibility checks and such here.
+ //
+ // If the source value already has a refined length then we'll
+ // transfer those refinements to the result, because conversion
+ // does not change length (aside from set element coalescing).
+ b := ret.Refine()
+ if targetTy.IsSetType() {
+ if sourceRange.LengthLowerBound() > 0 {
+ // If the source has at least one element then the result
+ // must always have at least one too, because value coalescing
+ // cannot totally empty the set.
+ b = b.CollectionLengthLowerBound(1)
+ }
+ } else {
+ b = b.CollectionLengthLowerBound(sourceRange.LengthLowerBound())
+ }
+ b = b.CollectionLengthUpperBound(sourceRange.LengthUpperBound())
+ return b.NewValue()
+ default:
+ return ret
+ }
+
+}
diff --git a/cty/convert/public_test.go b/cty/convert/public_test.go
index 44be53b..85e9219 100644
--- a/cty/convert/public_test.go
+++ b/cty/convert/public_test.go
@@ -1615,6 +1615,161 @@ func TestConvert(t *testing.T) {
})),
}),
},
+
+ // Object to map refinements
+ {
+ Value: cty.UnknownVal(cty.EmptyObject),
+ Type: cty.Map(cty.String),
+ Want: cty.UnknownVal(cty.Map(cty.String)).Refine().
+ CollectionLength(0).
+ NewValue(),
+ },
+ {
+ Value: cty.UnknownVal(cty.EmptyObject).RefineNotNull(),
+ Type: cty.Map(cty.String),
+ Want: cty.MapValEmpty(cty.String),
+ },
+ {
+ Value: cty.UnknownVal(cty.Object(map[string]cty.Type{"a": cty.String})),
+ Type: cty.Map(cty.String),
+ Want: cty.UnknownVal(cty.Map(cty.String)).Refine().
+ CollectionLength(1).
+ NewValue(),
+ },
+ {
+ Value: cty.UnknownVal(cty.Object(map[string]cty.Type{"a": cty.String})).RefineNotNull(),
+ Type: cty.Map(cty.String),
+ Want: cty.UnknownVal(cty.Map(cty.String)).Refine().
+ NotNull().
+ CollectionLength(1).
+ NewValue(),
+ },
+
+ // Tuple to list refinements
+ {
+ Value: cty.UnknownVal(cty.EmptyTuple),
+ Type: cty.List(cty.String),
+ Want: cty.UnknownVal(cty.List(cty.String)).Refine().
+ CollectionLength(0).
+ NewValue(),
+ },
+ {
+ Value: cty.UnknownVal(cty.EmptyTuple).RefineNotNull(),
+ Type: cty.List(cty.String),
+ Want: cty.ListValEmpty(cty.String),
+ },
+ {
+ Value: cty.UnknownVal(cty.Tuple([]cty.Type{cty.String})),
+ Type: cty.List(cty.String),
+ Want: cty.UnknownVal(cty.List(cty.String)).Refine().
+ CollectionLength(1).
+ NewValue(),
+ },
+ {
+ Value: cty.UnknownVal(cty.Tuple([]cty.Type{cty.String})).RefineNotNull(),
+ Type: cty.List(cty.String),
+ Want: cty.ListVal([]cty.Value{cty.UnknownVal(cty.String)}),
+ },
+
+ // Tuple to set refinements
+ {
+ Value: cty.UnknownVal(cty.EmptyTuple),
+ Type: cty.Set(cty.String),
+ Want: cty.UnknownVal(cty.Set(cty.String)).Refine().
+ CollectionLength(0).
+ NewValue(),
+ },
+ {
+ Value: cty.UnknownVal(cty.EmptyTuple).RefineNotNull(),
+ Type: cty.Set(cty.String),
+ Want: cty.SetValEmpty(cty.String),
+ },
+ {
+ Value: cty.UnknownVal(cty.Tuple([]cty.Type{cty.String})),
+ Type: cty.Set(cty.String),
+ Want: cty.UnknownVal(cty.Set(cty.String)).Refine().
+ CollectionLength(1).
+ NewValue(),
+ },
+ {
+ Value: cty.UnknownVal(cty.Tuple([]cty.Type{cty.String})).RefineNotNull(),
+ Type: cty.Set(cty.String),
+ Want: cty.SetVal([]cty.Value{cty.UnknownVal(cty.String)}),
+ },
+ {
+ Value: cty.UnknownVal(cty.Tuple([]cty.Type{cty.String, cty.String})),
+ Type: cty.Set(cty.String),
+ Want: cty.UnknownVal(cty.Set(cty.String)).Refine().
+ CollectionLengthLowerBound(1).
+ CollectionLengthUpperBound(2).
+ NewValue(),
+ },
+ {
+ Value: cty.UnknownVal(cty.Tuple([]cty.Type{cty.String, cty.String})).RefineNotNull(),
+ Type: cty.Set(cty.String),
+ Want: cty.UnknownVal(cty.Set(cty.String)).Refine().
+ NotNull().
+ CollectionLengthLowerBound(1).
+ CollectionLengthUpperBound(2).
+ NewValue(),
+ },
+
+ // Collection to collection refinements
+ {
+ Value: cty.UnknownVal(cty.List(cty.String)).Refine().
+ CollectionLengthLowerBound(2).
+ CollectionLengthUpperBound(4).
+ NewValue(),
+ Type: cty.Set(cty.String),
+ Want: cty.UnknownVal(cty.Set(cty.String)).Refine().
+ CollectionLengthLowerBound(1).
+ CollectionLengthUpperBound(4).
+ NewValue(),
+ },
+ {
+ Value: cty.UnknownVal(cty.List(cty.String)).Refine().
+ NotNull().
+ CollectionLengthLowerBound(2).
+ CollectionLengthUpperBound(4).
+ NewValue(),
+ Type: cty.Set(cty.String),
+ Want: cty.UnknownVal(cty.Set(cty.String)).Refine().
+ NotNull().
+ CollectionLengthLowerBound(1).
+ CollectionLengthUpperBound(4).
+ NewValue(),
+ },
+ {
+ Value: cty.UnknownVal(cty.Set(cty.String)).Refine().
+ CollectionLengthLowerBound(2).
+ CollectionLengthUpperBound(4).
+ NewValue(),
+ Type: cty.List(cty.String),
+ Want: cty.UnknownVal(cty.List(cty.String)).Refine().
+ CollectionLengthLowerBound(2).
+ CollectionLengthUpperBound(4).
+ NewValue(),
+ },
+ {
+ Value: cty.UnknownVal(cty.Set(cty.String)).Refine().
+ NotNull().
+ CollectionLengthLowerBound(2).
+ CollectionLengthUpperBound(4).
+ NewValue(),
+ Type: cty.List(cty.String),
+ Want: cty.UnknownVal(cty.List(cty.String)).Refine().
+ NotNull().
+ CollectionLengthLowerBound(2).
+ CollectionLengthUpperBound(4).
+ NewValue(),
+ },
+
+ // General unknown value refinements
+ {
+ Value: cty.UnknownVal(cty.Bool).RefineNotNull(),
+ Type: cty.String,
+ Want: cty.UnknownVal(cty.String).RefineNotNull(),
+ },
}
for _, test := range tests {
diff --git a/cty/ctystrings/doc.go b/cty/ctystrings/doc.go
new file mode 100644
index 0000000..0ea7f98
--- /dev/null
+++ b/cty/ctystrings/doc.go
@@ -0,0 +1,26 @@
+// Package ctystrings is a collection of string manipulation utilities which
+// intend to help application developers implement string-manipulation
+// functionality in a way that respects the cty model of strings, even when
+// they are working in the realm of Go strings.
+//
+// cty strings are, internally, NFC-normalized as defined in Unicode Standard
+// Annex #15 and encoded as UTF-8.
+//
+// When working with [cty.Value] of string type cty manages this
+// automatically as an implementation detail, but when applications call
+// [Value.AsString] they will receive a value that has been subjected to that
+// normalization, and so may need to take that normalization into account when
+// manipulating the resulting string or comparing it with other Go strings
+// that did not originate in a [cty.Value].
+//
+// Although the core representation of [cty.String] only considers whole
+// strings, it's also conventional in other locations such as the standard
+// library functions to consider strings as being sequences of grapheme
+// clusters as defined by Unicode Standard Annex #29, which adds further
+// rules about combining multiple consecutive codepoints together into a
+// single user-percieved character. Functions that work with substrings should
+// always use grapheme clusters as their smallest unit of splitting strings,
+// and never break strings in the middle of a grapheme cluster. The functions
+// in this package respect that convention unless otherwise stated in their
+// documentation.
+package ctystrings
diff --git a/cty/ctystrings/normalize.go b/cty/ctystrings/normalize.go
new file mode 100644
index 0000000..9b3bce9
--- /dev/null
+++ b/cty/ctystrings/normalize.go
@@ -0,0 +1,14 @@
+package ctystrings
+
+import (
+ "golang.org/x/text/unicode/norm"
+)
+
+// Normalize applies NFC normalization to the given string, returning the
+// transformed string.
+//
+// This function achieves the same effect as wrapping a string in a value
+// using [cty.StringVal] and then unwrapping it again using [Value.AsString].
+func Normalize(str string) string {
+ return norm.NFC.String(str)
+}
diff --git a/cty/ctystrings/prefix.go b/cty/ctystrings/prefix.go
new file mode 100644
index 0000000..1d9f5c5
--- /dev/null
+++ b/cty/ctystrings/prefix.go
@@ -0,0 +1,139 @@
+package ctystrings
+
+import (
+ "fmt"
+ "unicode/utf8"
+
+ "github.com/apparentlymart/go-textseg/v13/textseg"
+ "golang.org/x/text/unicode/norm"
+)
+
+// SafeKnownPrefix takes a string intended to represent a known prefix of
+// another string and modifies it so that it would be safe to use with
+// byte-based prefix matching against another NFC-normalized string. It
+// also takes into account grapheme cluster boundaries and trims off any
+// suffix that could potentially be an incomplete grapheme cluster.
+//
+// Specifically, SafeKnownPrefix first applies NFC normalization to the prefix
+// and then trims off one or more characters from the end of the string which
+// could potentially be transformed into a different character if another
+// string were appended to it. For example, a trailing latin letter will
+// typically be trimmed because appending a combining diacritic mark would
+// transform it into a different character.
+//
+// This transformation is important whenever the remainder of the string is
+// arbitrary user input not directly controlled by the application. If an
+// application can guarantee that the remainder of the string will not begin
+// with combining marks then it is safe to instead just normalize the prefix
+// string with [Normalize].
+//
+// Note that this function only takes into account normalization boundaries
+// and does _not_ take into account grapheme cluster boundaries as defined
+// by Unicode Standard Annex #29.
+func SafeKnownPrefix(prefix string) string {
+ prefix = Normalize(prefix)
+
+ // Our starting approach here is essentially what a streaming parser would
+ // do when consuming a Unicode string in chunks and needing to determine
+ // what prefix of the current buffer is safe to process without waiting for
+ // more information, which is described in TR15 section 13.1
+ // "Buffering with Unicode Normalization":
+ // https://unicode.org/reports/tr15/#Buffering_with_Unicode_Normalization
+ //
+ // The general idea here is to find the last character in the string that
+ // could potentially start a sequence of codepoints that would combine
+ // together, and then truncate the string to exclude that character and
+ // everything after it.
+
+ form := norm.NFC
+ lastBoundary := form.LastBoundary([]byte(prefix))
+ if lastBoundary != -1 && lastBoundary != len(prefix) {
+ prefix = prefix[:lastBoundary]
+ // If we get here then we've already shortened the prefix and so
+ // further analysis below is unnecessary because it would be relying
+ // on an incomplete prefix anyway.
+ return prefix
+ }
+
+ // Now we'll use the textseg package's grapheme cluster scanner to scan
+ // as far through the string as we can without the scanner telling us
+ // that it would need more bytes to decide.
+ //
+ // This step is conservative because the grapheme cluster rules are not
+ // designed with prefix-matching in mind. In the base case we'll just
+ // always discard the last grapheme cluster, although we do have some
+ // special cases for trailing codepoints that can't possibly combine with
+ // subsequent codepoints to form a single grapheme cluster and which seem
+ // likely to arise often in practical use.
+ remain := []byte(prefix)
+ prevBoundary := 0
+ thisBoundary := 0
+ for len(remain) > 0 {
+ advance, _, err := textseg.ScanGraphemeClusters(remain, false)
+ if err != nil {
+ // ScanGraphemeClusters should never return an error because
+ // any sequence of valid UTF-8 encodings is valid input.
+ panic(fmt.Sprintf("textseg.ScanGraphemeClusters returned error: %s", err))
+ }
+ if advance == 0 {
+ // If we have at least one byte remaining but the scanner cannot
+ // advance then that means the remainder might be an incomplete
+ // grapheme cluster and so we need to stop here, discarding the
+ // rest of the input. However, we do now know that we can safely
+ // include what we found on the previous iteration of this loop.
+ prevBoundary = thisBoundary
+ break
+ }
+ prevBoundary = thisBoundary
+ thisBoundary += advance
+ remain = remain[advance:]
+ }
+
+ // This is our heuristic for detecting cases where we can be sure that
+ // the above algorithm was too conservative because the last segment
+ // we found is definitely not subject to the grapheme cluster "do not split"
+ // rules.
+ suspect := prefix[prevBoundary:thisBoundary]
+ if sequenceMustEndGraphemeCluster(suspect) {
+ prevBoundary = thisBoundary
+ }
+
+ return prefix[:prevBoundary]
+}
+
+// sequenceMustEndGraphemeCluster is a heuristic we use to avoid discarding
+// the final grapheme cluster of a prefix in SafeKnownPrefix by recognizing
+// that a particular sequence is one known to not be subject to any of
+// the UAX29 "do not break" rules.
+//
+// If this function returns true then it is safe to include the given byte
+// sequence at the end of a safe prefix. Otherwise we don't know whether or
+// not it is safe.
+func sequenceMustEndGraphemeCluster(s string) bool {
+ // For now we're only considering sequences that represent a single
+ // codepoint. We'll assume that any sequence of two or more codepoints
+ // that could be a grapheme cluster might be extendable.
+ if utf8.RuneCountInString(s) != 1 {
+ return false
+ }
+
+ r, _ := utf8.DecodeRuneInString(s)
+
+ // Our initial ruleset is focused on characters that are commonly used
+ // as delimiters in text intended for both human and machine use, such
+ // as JSON documents.
+ //
+ // We don't include any letters or digits of any script here intentionally
+ // because those are the ones most likely to be subject to combining rules
+ // in either current or future Unicode specifications.
+ //
+ // We can safely grow this set over time, but we should be very careful
+ // about shrinking it because it could cause value refinements to loosen
+ // and thus cause results that were once known to become unknown.
+ switch r {
+ case '-', '_', ':', ';', '/', '\\', ',', '.', '(', ')', '{', '}', '[', ']', '|', '?', '!', '~', ' ', '\t', '@', '#', '$', '%', '^', '&', '*', '+', '"', '\'':
+ return true
+ default:
+ return false
+ }
+}
diff --git a/cty/ctystrings/prefix_test.go b/cty/ctystrings/prefix_test.go
new file mode 100644
index 0000000..7ca3024
--- /dev/null
+++ b/cty/ctystrings/prefix_test.go
@@ -0,0 +1,208 @@
+package ctystrings
+
+import (
+ "testing"
+)
+
+func TestSafeKnownPrefix(t *testing.T) {
+ tests := []struct {
+ Input, Want string
+ }{
+ // NOTE: Under future improvements to SafeKnownPrefix the "Want"
+ // results for all of these tests can safely get longer, thereby
+ // describing a more precise constraint, but we should avoid making
+ // them shorter because that will weaken existing constraints from
+ // older versions.
+ // (We might make exceptions for behaviors that are found to be
+ // clearly wrong, but consider the consequences carefully.)
+
+ {
+ "",
+ "",
+ },
+ {
+ "a",
+ "", // The "a" is discarded because it might combine with diacritics to follow
+ },
+ {
+ "boo",
+ "bo", // The final o is discarded because it might combine with diacritics to follow
+ },
+ {
+ "boop\r",
+ "boop", // The final \r is discarded because it could combine with \r\n to produce a single grapheme cluster
+ },
+ {
+ "hello 가",
+ "hello ", // Hangul syllables can combine arbitrarily, so we must trim of trailing ones
+ },
+ {
+ "hello 🤷🏽♂️",
+ "hello ", // We conservatively trim the whole emoji sequence because other emoji modifiers might come in later unicode specs
+ },
+ {
+ "hello 🤷🏽♂️ ",
+ "hello 🤷🏽♂️ ", // A subsequent character avoids the need to trim
+ },
+ {
+ "hello 🤷",
+ "hello ", // "Person Shrugging" can potentially combine with subsequent skin tone modifiers or ZWJ followed by gender presentation modifiers
+ },
+ {
+ "hello 🤷 ",
+ "hello 🤷 ", // A subsequent character avoids the need to trim
+ },
+ {
+ "hello 🤷\u200d", // U+200D is "zero width joiner"
+ "hello ", // The "Person Shrugging" followed by zero with joiner anticipates a subsequent modifier to join with
+ },
+ {
+ "hello \U0001f1e6", // This is the beginning of a "regional indicator symbol", which are supposed to appear in pairs but we only have one here
+ "hello ", // The symbol was discarded because we can't know what character it represents until we have both parts
+ },
+ {
+ "hello \U0001f1e6\U0001f1e6", // This is a regional indicator symbol "AA", which happens to be Aruba but it's not important exactly which country we're encoding
+ "hello ", // The text segmentation spec allows any number of consecutive regional indicators, so we must always discard any number of them at the end.
+ },
+ {
+ "hello \U0001f1e6\U0001f1e6 ",
+ "hello \U0001f1e6\U0001f1e6 ", // A subsequent character avoids the need to trim
+ },
+
+ // The following all rely on our additional heuristic about certain
+ // commonly-used delimiters that we know can never be the beginning
+ // of a combined grapheme cluster sequence. We make these exceptions
+ // because cty tends to be used more often for constructing strings
+ // for use by machines than for constructing text for human consumption.
+ {
+ "ami-", // e.g. prefix of an Amazon EC2 object identifier
+ "ami-",
+ },
+ {
+ "foo_", // e.g. prefix of a variable name
+ "foo_",
+ },
+ {
+ `{"foo":`, // e.g. prefix of a JSON object
+ `{"foo":`,
+ },
+ {
+ `beep();`, // e.g. prefix of a program in a C-like language?
+ `beep();`,
+ },
+ {
+ `https://`, // e.g. prefix of a URL with a known scheme
+ `https://`,
+ },
+ {
+ `c:\`, // e.g. windows filesystem path with a known drive letter
+ `c:\`,
+ },
+ {
+ `["foo",`, // e.g. prefix of a JSON document that includes a partially-known array
+ `["foo",`,
+ },
+ {
+ `foo.bar.`, // e.g. prefix of a traversal through attributes
+ `foo.bar.`,
+ },
+ {
+ `beep(`, // e.g. prefix of a program in a C-like language?
+ `beep(`,
+ },
+ {
+ `beep()`, // e.g. prefix of a program in a C-like language?
+ `beep()`,
+ },
+ {
+ `{`, // e.g. prefix of a JSON object
+ `{`,
+ },
+ {
+ `[{}`, // e.g. fragment of JSON
+ `[{}`,
+ },
+ {
+ `[`, // e.g. prefix of a JSON array
+ `[`,
+ },
+ {
+ `[[]`, // e.g. fragment of JSON
+ `[[]`,
+ },
+ {
+ `whatever |`, // e.g. partial Unix-style command line
+ `whatever |`,
+ },
+ {
+ `https://example.com/foo?`, // e.g. prefix of a URL with a query string
+ `https://example.com/foo?`,
+ },
+ {
+ `boop!`, // dunno but seems weird to have ? without !
+ `boop!`,
+ },
+ {
+ `ls ~`, // A reference to somebody's home directory
+ `ls ~`,
+ },
+ {
+ `a `, // A space always disambiguates whether our suffix is safe
+ `a `,
+ },
+ {
+ "a\t", // A tab always disambiguates whether our suffix is safe
+ "a\t",
+ },
+ {
+ `username@`, // e.g. incomplete email address
+ `username@`,
+ },
+ {
+ `#`, // e.g. start of a single-linecomment in some machine languages, or a "hashtag"
+ `#`,
+ },
+ {
+ `print $`, // e.g. start of a reference to a Perl scalar
+ `print $`,
+ },
+ {
+ `print %`, // e.g. start of a reference to a Perl hash
+ `print %`,
+ },
+ {
+ `^`, // e.g. start of a pessimistic version constraint in some version constraint syntaxes
+ `^`,
+ },
+ {
+ `foo(&`, // e.g. the "address of" operator in some programming languages
+ `foo(&`,
+ },
+ {
+ `foo *`, // e.g. multiplying by something
+ `foo *`,
+ },
+ {
+ `foo +`, // e.g. addition
+ `foo +`,
+ },
+ {
+ `["`, // e.g. we know it's a JSON string but we don't know the content yet
+ `["`,
+ },
+ {
+ `['`, // e.g. a string in a JSON-like language that also supports single quotes!
+ `['`,
+ },
+ }
+
+ for _, test := range tests {
+ t.Run(test.Input, func(t *testing.T) {
+ got := SafeKnownPrefix(test.Input)
+
+ if got != test.Want {
+ t.Errorf("wrong result\ninput: %q\ngot: %q\nwant: %q", test.Input, got, test.Want)
+ }
+ })
+ }
+}
diff --git a/cty/function/function.go b/cty/function/function.go
index c4d99f6..6fc9682 100644
--- a/cty/function/function.go
+++ b/cty/function/function.go
@@ -39,6 +39,19 @@ type Spec struct {
// depending on its arguments.
Type TypeFunc
+ // RefineResult is an optional callback for describing additional
+ // refinements for the result value beyond what can be described using
+ // a type constraint.
+ //
+ // A refinement callback should always return the same builder it was
+ // given, typically after modifying it using the methods of
+ // [cty.RefinementBuilder].
+ //
+ // Any refinements described by this callback must hold for the entire
+ // range of results from the function. For refinements that only apply
+ // to certain results, use direct refinement within [Impl] instead.
+ RefineResult func(*cty.RefinementBuilder) *cty.RefinementBuilder
+
// Impl is the ImplFunc that implements the function's behavior.
//
// Functions are expected to behave as pure functions, and not create
@@ -109,20 +122,13 @@ func (f Function) ReturnType(argTypes []cty.Type) (cty.Type, error) {
return f.ReturnTypeForValues(vals)
}
-// ReturnTypeForValues is similar to ReturnType but can be used if the caller
-// already knows the values of some or all of the arguments, in which case
-// the function may be able to determine a more definite result if its
-// return type depends on the argument *values*.
-//
-// For any arguments whose values are not known, pass an Unknown value of
-// the appropriate type.
-func (f Function) ReturnTypeForValues(args []cty.Value) (ty cty.Type, err error) {
+func (f Function) returnTypeForValues(args []cty.Value) (ty cty.Type, dynTypedArgs bool, err error) {
var posArgs []cty.Value
var varArgs []cty.Value
if f.spec.VarParam == nil {
if len(args) != len(f.spec.Params) {
- return cty.Type{}, fmt.Errorf(
+ return cty.Type{}, false, fmt.Errorf(
"wrong number of arguments (%d required; %d given)",
len(f.spec.Params), len(args),
)
@@ -132,7 +138,7 @@ func (f Function) ReturnTypeForValues(args []cty.Value) (ty cty.Type, err error)
varArgs = nil
} else {
if len(args) < len(f.spec.Params) {
- return cty.Type{}, fmt.Errorf(
+ return cty.Type{}, false, fmt.Errorf(
"wrong number of arguments (at least %d required; %d given)",
len(f.spec.Params), len(args),
)
@@ -161,7 +167,7 @@ func (f Function) ReturnTypeForValues(args []cty.Value) (ty cty.Type, err error)
}
if val.IsNull() && !spec.AllowNull {
- return cty.Type{}, NewArgErrorf(i, "argument must not be null")
+ return cty.Type{}, false, NewArgErrorf(i, "argument must not be null")
}
// AllowUnknown is ignored for type-checking, since we expect to be
@@ -171,13 +177,13 @@ func (f Function) ReturnTypeForValues(args []cty.Value) (ty cty.Type, err error)
if val.Type() == cty.DynamicPseudoType {
if !spec.AllowDynamicType {
- return cty.DynamicPseudoType, nil
+ return cty.DynamicPseudoType, true, nil
}
} else if errs := val.Type().TestConformance(spec.Type); errs != nil {
// For now we'll just return the first error in the set, since
// we don't have a good way to return the whole list here.
// Would be good to do something better at some point...
- return cty.Type{}, NewArgError(i, errs[0])
+ return cty.Type{}, false, NewArgError(i, errs[0])
}
}
@@ -196,18 +202,18 @@ func (f Function) ReturnTypeForValues(args []cty.Value) (ty cty.Type, err error)
}
if val.IsNull() && !spec.AllowNull {
- return cty.Type{}, NewArgErrorf(realI, "argument must not be null")
+ return cty.Type{}, false, NewArgErrorf(realI, "argument must not be null")
}
if val.Type() == cty.DynamicPseudoType {
if !spec.AllowDynamicType {
- return cty.DynamicPseudoType, nil
+ return cty.DynamicPseudoType, true, nil
}
} else if errs := val.Type().TestConformance(spec.Type); errs != nil {
// For now we'll just return the first error in the set, since
// we don't have a good way to return the whole list here.
// Would be good to do something better at some point...
- return cty.Type{}, NewArgError(i, errs[0])
+ return cty.Type{}, false, NewArgError(i, errs[0])
}
}
}
@@ -221,17 +227,53 @@ func (f Function) ReturnTypeForValues(args []cty.Value) (ty cty.Type, err error)
}
}()
- return f.spec.Type(args)
+ ty, err = f.spec.Type(args)
+ return ty, false, err
+}
+
+// ReturnTypeForValues is similar to ReturnType but can be used if the caller
+// already knows the values of some or all of the arguments, in which case
+// the function may be able to determine a more definite result if its
+// return type depends on the argument *values*.
+//
+// For any arguments whose values are not known, pass an Unknown value of
+// the appropriate type.
+func (f Function) ReturnTypeForValues(args []cty.Value) (ty cty.Type, err error) {
+ ty, _, err = f.returnTypeForValues(args)
+ return ty, err
}
// Call actually calls the function with the given arguments, which must
// conform to the function's parameter specification or an error will be
// returned.
func (f Function) Call(args []cty.Value) (val cty.Value, err error) {
- expectedType, err := f.ReturnTypeForValues(args)
+ expectedType, dynTypeArgs, err := f.returnTypeForValues(args)
if err != nil {
return cty.NilVal, err
}
+ if dynTypeArgs {
+ // returnTypeForValues sets this if any argument was inexactly typed
+ // and the corresponding parameter did not indicate it could deal with
+ // that. In that case we also avoid calling the implementation function
+ // because it will also typically not be ready to deal with that case.
+ return cty.UnknownVal(expectedType), nil
+ }
+
+ if refineResult := f.spec.RefineResult; refineResult != nil {
+ // If this function has a refinement callback then we'll refine
+ // our result value in the same way regardless of how we return.
+ // It's the function author's responsibility to ensure that the
+ // refinements they specify are valid for the full range of possible
+ // return values from the function. If not, this will panic when
+ // detecting an inconsistency.
+ defer func() {
+ if val != cty.NilVal {
+ if val.IsKnown() || val.Type() != cty.DynamicPseudoType {
+ val = val.RefineWith(refineResult)
+ }
+ }
+ }()
+ }
// Type checking already dealt with most situations relating to our
// parameter specification, but we still need to deal with unknown
diff --git a/cty/function/stdlib/bool.go b/cty/function/stdlib/bool.go
index 8192d8c..2826bf6 100644
--- a/cty/function/stdlib/bool.go
+++ b/cty/function/stdlib/bool.go
@@ -15,7 +15,8 @@ var NotFunc = function.New(&function.Spec{
AllowMarked: true,
},
},
- Type: function.StaticReturnType(cty.Bool),
+ Type: function.StaticReturnType(cty.Bool),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
return args[0].Not(), nil
},
@@ -37,7 +38,8 @@ var AndFunc = function.New(&function.Spec{
AllowMarked: true,
},
},
- Type: function.StaticReturnType(cty.Bool),
+ Type: function.StaticReturnType(cty.Bool),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
return args[0].And(args[1]), nil
},
@@ -59,7 +61,8 @@ var OrFunc = function.New(&function.Spec{
AllowMarked: true,
},
},
- Type: function.StaticReturnType(cty.Bool),
+ Type: function.StaticReturnType(cty.Bool),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
return args[0].Or(args[1]), nil
},
diff --git a/cty/function/stdlib/bool_test.go b/cty/function/stdlib/bool_test.go
index 9a2179f..e170b2e 100644
--- a/cty/function/stdlib/bool_test.go
+++ b/cty/function/stdlib/bool_test.go
@@ -22,11 +22,11 @@ func TestNot(t *testing.T) {
},
{
cty.UnknownVal(cty.Bool),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.True.Mark(1),
@@ -78,22 +78,22 @@ func TestAnd(t *testing.T) {
{
cty.True,
cty.UnknownVal(cty.Bool),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.UnknownVal(cty.Bool),
cty.UnknownVal(cty.Bool),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.True,
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
}
@@ -141,22 +141,22 @@ func TestOr(t *testing.T) {
{
cty.True,
cty.UnknownVal(cty.Bool),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.UnknownVal(cty.Bool),
cty.UnknownVal(cty.Bool),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.True,
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
}
diff --git a/cty/function/stdlib/bytes.go b/cty/function/stdlib/bytes.go
index 3fe600f..fe67e6f 100644
--- a/cty/function/stdlib/bytes.go
+++ b/cty/function/stdlib/bytes.go
@@ -38,7 +38,8 @@ var BytesLenFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
bufPtr := args[0].EncapsulatedValue().(*[]byte)
return cty.NumberIntVal(int64(len(*bufPtr))), nil
@@ -65,7 +66,8 @@ var BytesSliceFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(Bytes),
+ Type: function.StaticReturnType(Bytes),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
bufPtr := args[0].EncapsulatedValue().(*[]byte)
diff --git a/cty/function/stdlib/collection.go b/cty/function/stdlib/collection.go
index 0573e74..1816bb9 100644
--- a/cty/function/stdlib/collection.go
+++ b/cty/function/stdlib/collection.go
@@ -32,6 +32,7 @@ var HasIndexFunc = function.New(&function.Spec{
}
return cty.Bool, nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
return args[0].HasIndex(args[1]), nil
},
@@ -114,6 +115,7 @@ var LengthFunc = function.New(&function.Spec{
Name: "collection",
Type: cty.DynamicPseudoType,
AllowDynamicType: true,
+ AllowUnknown: true,
AllowMarked: true,
},
},
@@ -124,6 +126,7 @@ var LengthFunc = function.New(&function.Spec{
}
return cty.Number, nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
return args[0].Length(), nil
},
@@ -251,6 +254,7 @@ var CoalesceListFunc = function.New(&function.Spec{
return last, nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
for _, arg := range args {
if !arg.IsKnown() {
@@ -283,7 +287,8 @@ var CompactFunc = function.New(&function.Spec{
Type: cty.List(cty.String),
},
},
- Type: function.StaticReturnType(cty.List(cty.String)),
+ Type: function.StaticReturnType(cty.List(cty.String)),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
listVal := args[0]
if !listVal.IsWhollyKnown() {
@@ -324,7 +329,8 @@ var ContainsFunc = function.New(&function.Spec{
Type: cty.DynamicPseudoType,
},
},
- Type: function.StaticReturnType(cty.Bool),
+ Type: function.StaticReturnType(cty.Bool),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
arg := args[0]
ty := arg.Type()
@@ -382,6 +388,7 @@ var DistinctFunc = function.New(&function.Spec{
Type: func(args []cty.Value) (cty.Type, error) {
return args[0].Type(), nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
listVal := args[0]
@@ -426,6 +433,7 @@ var ChunklistFunc = function.New(&function.Spec{
Type: func(args []cty.Value) (cty.Type, error) {
return cty.List(args[0].Type()), nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
listVal := args[0]
sizeVal := args[1]
@@ -513,6 +521,7 @@ var FlattenFunc = function.New(&function.Spec{
}
return cty.Tuple(tys), nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
inputList := args[0]
@@ -611,6 +620,7 @@ var KeysFunc = function.New(&function.Spec{
return cty.DynamicPseudoType, function.NewArgErrorf(0, "must have map or object type")
}
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
// We must unmark the value before we can use ElementIterator on it, and
// then re-apply the same marks (possibly none) when we return. Since we
@@ -832,6 +842,7 @@ var MergeFunc = function.New(&function.Spec{
return cty.Object(attrs), nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
outputMap := make(map[string]cty.Value)
var markses []cty.ValueMarks // remember any marked maps/objects we find
@@ -891,6 +902,7 @@ var ReverseListFunc = function.New(&function.Spec{
return cty.NilType, function.NewArgErrorf(0, "can only reverse list or tuple values, not %s", argTy.FriendlyName())
}
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
in, marks := args[0].Unmark()
inVals := in.AsValueSlice()
@@ -919,10 +931,11 @@ var SetProductFunc = function.New(&function.Spec{
Description: `Calculates the cartesian product of two or more sets.`,
Params: []function.Parameter{},
VarParam: &function.Parameter{
- Name: "sets",
- Description: "The sets to consider. Also accepts lists and tuples, and if all arguments are of list or tuple type then the result will preserve the input ordering",
- Type: cty.DynamicPseudoType,
- AllowMarked: true,
+ Name: "sets",
+ Description: "The sets to consider. Also accepts lists and tuples, and if all arguments are of list or tuple type then the result will preserve the input ordering",
+ Type: cty.DynamicPseudoType,
+ AllowMarked: true,
+ AllowUnknown: true,
},
Type: func(args []cty.Value) (retType cty.Type, err error) {
if len(args) < 2 {
@@ -964,6 +977,7 @@ var SetProductFunc = function.New(&function.Spec{
}
return cty.Set(cty.Tuple(elemTys)), nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
ety := retType.ElementType()
var retMarks cty.ValueMarks
@@ -976,7 +990,7 @@ var SetProductFunc = function.New(&function.Spec{
// Continue processing after we find an argument with unknown
// length to ensure that we cover all the marks
- if !arg.Length().IsKnown() {
+ if !(arg.IsKnown() && arg.Length().IsKnown()) {
hasUnknownLength = true
continue
}
@@ -988,7 +1002,62 @@ var SetProductFunc = function.New(&function.Spec{
}
if hasUnknownLength {
- return cty.UnknownVal(retType).WithMarks(retMarks), nil
+ defer func() {
+ // We're definitely going to return from somewhere in this
+ // branch and however we do it we must reapply the marks
+ // on the way out.
+ ret = ret.WithMarks(retMarks)
+ }()
+ ret := cty.UnknownVal(retType)
+
+ // Even if we don't know the exact length we may be able to
+ // constrain the upper and lower bounds of the resulting length.
+ maxLength := 1
+ for _, arg := range args {
+ arg, _ := arg.Unmark() // safe to discard marks because "retMarks" already contains them all
+ argRng := arg.Range()
+ ty := argRng.TypeConstraint()
+ var argMaxLen int
+ if ty.IsCollectionType() {
+ argMaxLen = argRng.LengthUpperBound()
+ } else if ty.IsTupleType() {
+ argMaxLen = ty.Length()
+ } else {
+ // Should not get here but if we do then we'll just
+ // bail out with an unrefined unknown value.
+ return ret, nil
+ }
+ // The upper bound of a totally-unrefined collection is
+ // math.MaxInt, which will quickly get us to integer overflow
+ // here, and so out of pragmatism we'll just impose a reasonable
+ // upper limit on what is a useful bound to track and return
+ // unrefined for unusually-large input.
+ if argMaxLen > 1024 { // arbitrarily-decided threshold
+ return ret, nil
+ }
+ maxLength *= argMaxLen
+ if maxLength > 2048 { // arbitrarily-decided threshold
+ return ret, nil
+ }
+ if maxLength < 0 { // Seems like we already overflowed, then.
+ return ret, nil
+ }
+ }
+
+ if maxLength == 0 {
+ // This refinement will typically allow the unknown value to
+ // collapse into a known empty collection.
+ ret = ret.Refine().CollectionLength(0).NewValue()
+ } else {
+ // If we know there's a nonzero maximum number of elements then
+ // set element coalescing cannot reduce to fewer than one
+ // element.
+ ret = ret.Refine().
+ CollectionLengthLowerBound(1).
+ CollectionLengthUpperBound(maxLength).
+ NewValue()
+ }
+ return ret, nil
}
if total == 0 {
@@ -1101,6 +1170,7 @@ var SliceFunc = function.New(&function.Spec{
}
return cty.Tuple(argTy.TupleElementTypes()[startIndex:endIndex]), nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
inputList, marks := args[0].Unmark()
@@ -1215,6 +1285,7 @@ var ValuesFunc = function.New(&function.Spec{
}
return cty.NilType, errors.New("values() requires a map as the first argument")
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
mapVar := args[0]
@@ -1303,6 +1374,7 @@ var ZipmapFunc = function.New(&function.Spec{
return cty.NilType, errors.New("values argument must be a list or tuple value")
}
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
keys := args[0]
values := args[1]
diff --git a/cty/function/stdlib/collection_test.go b/cty/function/stdlib/collection_test.go
index eac92d5..b6b9b6b 100644
--- a/cty/function/stdlib/collection_test.go
+++ b/cty/function/stdlib/collection_test.go
@@ -57,22 +57,22 @@ func TestHasIndex(t *testing.T) {
{
cty.ListValEmpty(cty.Number),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.UnknownVal(cty.List(cty.Bool)),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.ListValEmpty(cty.Number),
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
}
@@ -107,7 +107,7 @@ func TestChunklist(t *testing.T) {
{
cty.UnknownVal(cty.List(cty.String)),
cty.NumberIntVal(2),
- cty.UnknownVal(cty.List(cty.List(cty.String))),
+ cty.UnknownVal(cty.List(cty.List(cty.String))).RefineNotNull(),
``,
},
{
@@ -359,7 +359,7 @@ func TestContains(t *testing.T) {
{
listWithUnknown,
cty.StringVal("orange"),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
false,
},
{
@@ -404,7 +404,7 @@ func TestContains(t *testing.T) {
cty.StringVal("fox"),
}),
cty.StringVal("quick"),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
false,
},
{ // set val
@@ -424,7 +424,7 @@ func TestContains(t *testing.T) {
cty.StringVal("fox"),
}),
cty.StringVal("quick"),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
false,
},
{ // nested unknown
@@ -436,7 +436,7 @@ func TestContains(t *testing.T) {
cty.ObjectVal(map[string]cty.Value{
"a": cty.StringVal("b"),
}),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
false,
},
{ // tuple val
@@ -557,7 +557,7 @@ func TestMerge(t *testing.T) {
"c": cty.StringVal("d"),
}),
},
- cty.UnknownVal(cty.Map(cty.String)),
+ cty.UnknownVal(cty.Map(cty.String)).RefineNotNull(),
false,
},
{ // handle dynamic unknown
@@ -947,7 +947,9 @@ func TestLength(t *testing.T) {
},
{
cty.SetVal([]cty.Value{cty.True, cty.UnknownVal(cty.Bool)}),
- cty.UnknownVal(cty.Number), // Don't know if the unknown in the input represents cty.True or cty.False
+ // Don't know if the unknown in the input represents cty.True or cty.False,
+ // so it may or may not coalesce with the one known value.
+ cty.UnknownVal(cty.Number).Refine().NotNull().NumberRangeInclusive(cty.NumberIntVal(1), cty.NumberIntVal(2)).NewValue(),
},
{
cty.SetVal([]cty.Value{cty.UnknownVal(cty.Bool)}),
@@ -971,11 +973,15 @@ func TestLength(t *testing.T) {
},
{
cty.UnknownVal(cty.List(cty.Bool)),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).Refine().NotNull().NumberRangeInclusive(cty.Zero, cty.NumberIntVal(int64(math.MaxInt))).NewValue(),
},
{
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).Refine().NotNull().NumberRangeInclusive(cty.Zero, cty.NumberIntVal(int64(math.MaxInt))).NewValue(),
+ },
+ {
+ cty.UnknownVal(cty.List(cty.Bool)).Refine().CollectionLengthUpperBound(2).NewValue(),
+ cty.UnknownVal(cty.Number).Refine().NotNull().NumberRangeInclusive(cty.Zero, cty.NumberIntVal(2)).NewValue(),
},
{ // Marked collections return a marked length
cty.ListVal([]cty.Value{
@@ -1389,7 +1395,7 @@ func TestValues(t *testing.T) {
},
{
cty.UnknownVal(cty.Map(cty.String)),
- cty.UnknownVal(cty.List(cty.String)),
+ cty.UnknownVal(cty.List(cty.String)).RefineNotNull(),
``,
},
{
@@ -1434,12 +1440,12 @@ func TestValues(t *testing.T) {
},
{
cty.UnknownVal(cty.EmptyObject),
- cty.UnknownVal(cty.EmptyTuple),
+ cty.UnknownVal(cty.EmptyTuple).RefineNotNull(),
``,
},
{
cty.UnknownVal(cty.Object(map[string]cty.Type{"a": cty.String})),
- cty.UnknownVal(cty.Tuple([]cty.Type{cty.String})),
+ cty.UnknownVal(cty.Tuple([]cty.Type{cty.String})).RefineNotNull(),
``,
},
{ // The object itself is not marked, just an inner attribute value.
@@ -1515,19 +1521,19 @@ func TestZipMap(t *testing.T) {
{
cty.UnknownVal(cty.List(cty.String)),
cty.UnknownVal(cty.List(cty.String)),
- cty.UnknownVal(cty.Map(cty.String)),
+ cty.UnknownVal(cty.Map(cty.String)).RefineNotNull(),
``,
},
{
cty.UnknownVal(cty.List(cty.String)),
cty.ListValEmpty(cty.String),
- cty.UnknownVal(cty.Map(cty.String)),
+ cty.UnknownVal(cty.Map(cty.String)).RefineNotNull(),
``,
},
{
cty.ListValEmpty(cty.String),
cty.UnknownVal(cty.List(cty.String)),
- cty.UnknownVal(cty.Map(cty.String)),
+ cty.UnknownVal(cty.Map(cty.String)).RefineNotNull(),
``,
},
{
@@ -1626,7 +1632,7 @@ func TestZipMap(t *testing.T) {
{
cty.ListValEmpty(cty.String),
cty.UnknownVal(cty.EmptyTuple),
- cty.UnknownVal(cty.EmptyObject),
+ cty.UnknownVal(cty.EmptyObject).RefineNotNull(),
``,
},
{
@@ -2489,7 +2495,109 @@ func TestSetproduct(t *testing.T) {
cty.SetVal([]cty.Value{cty.StringVal("x"), cty.UnknownVal(cty.String)}).Mark("a"),
cty.SetVal([]cty.Value{cty.True, cty.False}).Mark("b"),
},
- cty.UnknownVal(cty.Set(cty.Tuple([]cty.Type{cty.String, cty.Bool}))).WithMarks(cty.NewValueMarks("a", "b")),
+ cty.UnknownVal(cty.Set(cty.Tuple([]cty.Type{cty.String, cty.Bool}))).RefineNotNull().WithMarks(cty.NewValueMarks("a", "b")),
+ ``,
+ },
+ {
+ []cty.Value{
+ cty.SetVal([]cty.Value{cty.True}),
+ cty.DynamicVal,
+ },
+ cty.DynamicVal,
+ ``,
+ },
+
+ // If the inputs have unknown lengths but have length refinements then
+ // we can potentially refine our unknown result too.
+ {
+ []cty.Value{
+ cty.UnknownVal(cty.Set(cty.String)).Refine().CollectionLengthUpperBound(2).NewValue(),
+ cty.UnknownVal(cty.Set(cty.Number)).Refine().CollectionLengthUpperBound(3).NewValue(),
+ },
+ cty.UnknownVal(cty.Set(cty.Tuple([]cty.Type{cty.String, cty.Number}))).Refine().
+ NotNull().
+ CollectionLengthLowerBound(1).
+ CollectionLengthUpperBound(6).
+ NewValue(),
+ ``,
+ },
+ {
+ []cty.Value{
+ cty.UnknownVal(cty.Set(cty.String)).Refine().CollectionLengthUpperBound(2).NewValue(),
+ cty.SetValEmpty(cty.Number),
+ },
+ cty.SetValEmpty(cty.Tuple([]cty.Type{cty.String, cty.Number})), // deduced from refinements
+ ``,
+ },
+ {
+ // If we have any input with a very large maximum element count then we'll
+ // just leave the result length unrefined to reduce the risk of integer overflow.
+ []cty.Value{
+ cty.UnknownVal(cty.Set(cty.String)).Refine().CollectionLengthUpperBound(2).NewValue(),
+ cty.UnknownVal(cty.Set(cty.Number)).Refine().CollectionLengthUpperBound(4096).NewValue(),
+ },
+ cty.UnknownVal(cty.Set(cty.Tuple([]cty.Type{cty.String, cty.Number}))).RefineNotNull(),
+ ``,
+ },
+ {
+ []cty.Value{
+ cty.UnknownVal(cty.List(cty.String)).Refine().CollectionLengthUpperBound(2).NewValue(),
+ cty.UnknownVal(cty.List(cty.Number)).Refine().CollectionLengthUpperBound(3).NewValue(),
+ },
+ // NOTE: When the result is a list rather than a set there is no
+ // coalescing and so we could potentially also calculate a more
+ // refined lower bound on the collection length, but since
+ // this function is primarily for sets for now we just accept a
+ // set-oriented refinement. If we find that it would be productive
+ // to further constrain the range of a list result then we can
+ // make this more precise later.
+ cty.UnknownVal(cty.List(cty.Tuple([]cty.Type{cty.String, cty.Number}))).Refine().
+ NotNull().
+ CollectionLengthLowerBound(1).
+ CollectionLengthUpperBound(6).
+ NewValue(),
+ ``,
+ },
+ {
+ []cty.Value{
+ cty.UnknownVal(cty.List(cty.String)).Refine().CollectionLengthUpperBound(2).NewValue(),
+ cty.ListValEmpty(cty.Number),
+ },
+ cty.ListValEmpty(cty.Tuple([]cty.Type{cty.String, cty.Number})), // deduced from refinements
+ ``,
+ },
+ {
+ []cty.Value{
+ cty.UnknownVal(cty.Tuple([]cty.Type{cty.String, cty.String})),
+ cty.UnknownVal(cty.Tuple([]cty.Type{cty.Number, cty.Number, cty.Number})),
+ },
+ // NOTE: When the result is a list rather than a set there is no
+ // coalescing and so we could potentially also calculate a more
+ // refined lower bound on the collection length, but since
+ // this function is primarily for sets for now we just accept a
+ // set-oriented refinement. If we find that it would be productive
+ // to further constrain the range of a list result then we can
+ // make this more precise later.
+ cty.UnknownVal(cty.List(cty.Tuple([]cty.Type{cty.String, cty.Number}))).Refine().
+ NotNull().
+ CollectionLengthLowerBound(1).
+ CollectionLengthUpperBound(6).
+ NewValue(),
+ ``,
+ },
+ {
+ []cty.Value{
+ cty.UnknownVal(cty.Tuple([]cty.Type{cty.String, cty.String})),
+ cty.EmptyTupleVal,
+ },
+ // NOTE: When the result is a list rather than a set there is no
+ // coalescing and so we could potentially also calculate a more
+ // refined lower bound on the collection length, but since
+ // this function is primarily for sets for now we just accept a
+ // set-oriented refinement. If we find that it would be productive
+ // to further constrain the range of a list result then we can
+ // make this more precise later.
+ cty.ListValEmpty(cty.Tuple([]cty.Type{cty.String, cty.DynamicPseudoType})),
``,
},
}
@@ -2539,7 +2647,7 @@ func TestReverseList(t *testing.T) {
},
{
cty.UnknownVal(cty.List(cty.String)),
- cty.UnknownVal(cty.List(cty.String)),
+ cty.UnknownVal(cty.List(cty.String)).RefineNotNull(),
``,
},
{ // marks on list elements
diff --git a/cty/function/stdlib/conversion.go b/cty/function/stdlib/conversion.go
index f61b534..5d06a45 100644
--- a/cty/function/stdlib/conversion.go
+++ b/cty/function/stdlib/conversion.go
@@ -87,3 +87,36 @@ func MakeToFunc(wantTy cty.Type) function.Function {
},
})
}
+
+// AssertNotNullFunc is a function which does nothing except return an error
+// if the argument given to it is null.
+//
+// This could be useful in some cases where the automatic refinment of
+// nullability isn't precise enough, because the result is guaranteed to not
+// be null and can therefore allow downstream comparisons to null to return
+// a known value even if the value is otherwise unknown.
+var AssertNotNullFunc = function.New(&function.Spec{
+ Description: "Returns the given value varbatim if it is non-null, or raises an error if it's null.",
+ Params: []function.Parameter{
+ {
+ Name: "v",
+ Type: cty.DynamicPseudoType,
+ // NOTE: We intentionally don't set AllowNull here, and so
+ // the function system will automatically reject a null argument
+ // for us before calling Impl.
+ },
+ },
+ Type: func(args []cty.Value) (cty.Type, error) {
+ return args[0].Type(), nil
+ },
+ RefineResult: refineNonNull,
+ Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
+ // Our argument doesn't set AllowNull: true, so we're guaranteed to
+ // have a non-null value in args[0].
+ return args[0], nil
+ },
+})
+
+func AssertNotNull(v cty.Value) (cty.Value, error) {
+ return AssertNotNullFunc.Call([]cty.Value{v})
+}
diff --git a/cty/function/stdlib/csv.go b/cty/function/stdlib/csv.go
index 20d82bc..e854e81 100644
--- a/cty/function/stdlib/csv.go
+++ b/cty/function/stdlib/csv.go
@@ -43,6 +43,7 @@ var CSVDecodeFunc = function.New(&function.Spec{
}
return cty.List(cty.Object(atys)), nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
ety := retType.ElementType()
atys := ety.AttributeTypes()
diff --git a/cty/function/stdlib/datetime.go b/cty/function/stdlib/datetime.go
index 6c0ee05..85f58d4 100644
--- a/cty/function/stdlib/datetime.go
+++ b/cty/function/stdlib/datetime.go
@@ -23,7 +23,8 @@ var FormatDateFunc = function.New(&function.Spec{
Type: cty.String,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
formatStr := args[0].AsString()
timeStr := args[1].AsString()
@@ -281,67 +282,6 @@ func FormatDate(format cty.Value, timestamp cty.Value) (cty.Value, error) {
return FormatDateFunc.Call([]cty.Value{format, timestamp})
}
-func parseTimestamp(ts string) (time.Time, error) {
- t, err := time.Parse(time.RFC3339, ts)
- if err != nil {
- switch err := err.(type) {
- case *time.ParseError:
- // If err is s time.ParseError then its string representation is not
- // appropriate since it relies on details of Go's strange date format
- // representation, which a caller of our functions is not expected
- // to be familiar with.
- //
- // Therefore we do some light transformation to get a more suitable
- // error that should make more sense to our callers. These are
- // still not awesome error messages, but at least they refer to
- // the timestamp portions by name rather than by Go's example
- // values.
- if err.LayoutElem == "" && err.ValueElem == "" && err.Message != "" {
- // For some reason err.Message is populated with a ": " prefix
- // by the time package.
- return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp%s", err.Message)
- }
- var what string
- switch err.LayoutElem {
- case "2006":
- what = "year"
- case "01":
- what = "month"
- case "02":
- what = "day of month"
- case "15":
- what = "hour"
- case "04":
- what = "minute"
- case "05":
- what = "second"
- case "Z07:00":
- what = "UTC offset"
- case "T":
- return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: missing required time introducer 'T'")
- case ":", "-":
- if err.ValueElem == "" {
- return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: end of string where %q is expected", err.LayoutElem)
- } else {
- return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: found %q where %q is expected", err.ValueElem, err.LayoutElem)
- }
- default:
- // Should never get here, because time.RFC3339 includes only the
- // above portions, but since that might change in future we'll
- // be robust here.
- what = "timestamp segment"
- }
- if err.ValueElem == "" {
- return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: end of string before %s", what)
- } else {
- return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: cannot use %q as %s", err.ValueElem, what)
- }
- }
- return time.Time{}, err
- }
- return t, nil
-}
-
// splitDataFormat is a bufio.SplitFunc used to tokenize a date format.
func splitDateFormat(data []byte, atEOF bool) (advance int, token []byte, err error) {
if len(data) == 0 {
@@ -418,6 +358,75 @@ func startsDateFormatVerb(b byte) bool {
return (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z')
}
+func parseTimestamp(ts string) (time.Time, error) {
+ t, err := parseStrictRFC3339(ts)
+ if err != nil {
+ switch err := err.(type) {
+ case *time.ParseError:
+ // If err is s time.ParseError then its string representation is not
+ // appropriate since it relies on details of Go's strange date format
+ // representation, which a caller of our functions is not expected
+ // to be familiar with.
+ //
+ // Therefore we do some light transformation to get a more suitable
+ // error that should make more sense to our callers. These are
+ // still not awesome error messages, but at least they refer to
+ // the timestamp portions by name rather than by Go's example
+ // values.
+ if err.LayoutElem == "" && err.ValueElem == "" && err.Message != "" {
+ // For some reason err.Message is populated with a ": " prefix
+ // by the time package.
+ return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp%s", err.Message)
+ }
+ var what string
+ switch err.LayoutElem {
+ case "2006":
+ what = "year"
+ case "01":
+ what = "month"
+ case "02":
+ what = "day of month"
+ case "15":
+ what = "hour"
+ case "04":
+ what = "minute"
+ case "05":
+ what = "second"
+ case "Z07:00":
+ what = "UTC offset"
+ case "T":
+ return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: missing required time introducer 'T'")
+ case ":", "-":
+ if err.ValueElem == "" {
+ return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: end of string where %q is expected", err.LayoutElem)
+ } else {
+ return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: found %q where %q is expected", err.ValueElem, err.LayoutElem)
+ }
+ default:
+ // Should never get here, because RFC3339 includes only the
+ // above portions.
+ what = "timestamp segment"
+ }
+ if err.ValueElem == "" {
+ return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: end of string before %s", what)
+ } else {
+ switch {
+ case what == "hour" && strings.Contains(err.ValueElem, ":"):
+ return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: hour must be between 0 and 23 inclusive")
+ case what == "hour" && len(err.ValueElem) != 2:
+ return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: hour must have exactly two digits")
+ case what == "minute" && len(err.ValueElem) != 2:
+ return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: minute must have exactly two digits")
+ default:
+ return time.Time{}, fmt.Errorf("not a valid RFC3339 timestamp: cannot use %q as %s", err.ValueElem, what)
+ }
+ }
+ }
+ return time.Time{}, err
+ }
+ return t, nil
+}
+
// TimeAdd adds a duration to a timestamp, returning a new timestamp.
//
// In the HCL language, timestamps are conventionally represented as
diff --git a/cty/function/stdlib/datetime_rfc3339.go b/cty/function/stdlib/datetime_rfc3339.go
new file mode 100644
index 0000000..687854f
--- /dev/null
+++ b/cty/function/stdlib/datetime_rfc3339.go
@@ -0,0 +1,219 @@
+package stdlib
+
+import (
+ "errors"
+ "strconv"
+ "time"
+)
+
+// This file inlines some RFC3339 parsing code that was added to the Go standard
+// library's "time" package during the Go 1.20 development period but then
+// reverted prior to release to follow the Go proposals process first.
+//
+// Our goal is to support only valid RFC3339 strings regardless of what version
+// of Go is being used, because the Go stdlib is just an implementation detail
+// of the cty stdlib and so these functions should not very their behavior
+// significantly due to being compiled against a different Go version.
+//
+// These inline copies of the code from upstream should likely stay here
+// indefinitely even if functionality like this _is_ accepted in a later version
+// of Go, because this now defines cty's definition of RFC3339 parsing as
+// intentionally independent of Go's.
+
+func parseStrictRFC3339(str string) (time.Time, error) {
+ t, ok := parseRFC3339(str)
+ if !ok {
+ // If parsing failed then we'll try to use time.Parse to gather up a
+ // helpful error object.
+ _, err := time.Parse(time.RFC3339, str)
+ if err != nil {
+ return time.Time{}, err
+ }
+
+ // The parse template syntax cannot correctly validate RFC 3339.
+ // Explicitly check for cases that Parse is unable to validate for.
+ // See https://go.dev/issue/54580.
+ num2 := func(str string) byte { return 10*(str[0]-'0') + (str[1] - '0') }
+ switch {
+ case str[len("2006-01-02T")+1] == ':': // hour must be two digits
+ return time.Time{}, &time.ParseError{
+ Layout: time.RFC3339,
+ Value: str,
+ LayoutElem: "15",
+ ValueElem: str[len("2006-01-02T"):][:1],
+ Message: ": hour must have two digits",
+ }
+ case str[len("2006-01-02T15:04:05")] == ',': // sub-second separator must be a period
+ return time.Time{}, &time.ParseError{
+ Layout: time.RFC3339,
+ Value: str,
+ LayoutElem: ".",
+ ValueElem: ",",
+ Message: ": sub-second separator must be a period",
+ }
+ case str[len(str)-1] != 'Z':
+ switch {
+ case num2(str[len(str)-len("07:00"):]) >= 24: // timezone hour must be in range
+ return time.Time{}, &time.ParseError{
+ Layout: time.RFC3339,
+ Value: str,
+ LayoutElem: "Z07:00",
+ ValueElem: str[len(str)-len("Z07:00"):],
+ Message: ": timezone hour out of range",
+ }
+ case num2(str[len(str)-len("00"):]) >= 60: // timezone minute must be in range
+ return time.Time{}, &time.ParseError{
+ Layout: time.RFC3339,
+ Value: str,
+ LayoutElem: "Z07:00",
+ ValueElem: str[len(str)-len("Z07:00"):],
+ Message: ": timezone minute out of range",
+ }
+ }
+ default: // unknown error; should not occur
+ return time.Time{}, &time.ParseError{
+ Layout: time.RFC3339,
+ Value: str,
+ LayoutElem: time.RFC3339,
+ ValueElem: str,
+ Message: "",
+ }
+ }
+ }
+ return t, nil
+}
+
+func parseRFC3339(s string) (time.Time, bool) {
+ // parseUint parses s as an unsigned decimal integer and
+ // verifies that it is within some range.
+ // If it is invalid or out-of-range,
+ // it sets ok to false and returns the min value.
+ ok := true
+ parseUint := func(s string, min, max int) (x int) {
+ for _, c := range []byte(s) {
+ if c < '0' || '9' < c {
+ ok = false
+ return min
+ }
+ x = x*10 + int(c) - '0'
+ }
+ if x < min || max < x {
+ ok = false
+ return min
+ }
+ return x
+ }
+
+ // Parse the date and time.
+ if len(s) < len("2006-01-02T15:04:05") {
+ return time.Time{}, false
+ }
+ year := parseUint(s[0:4], 0, 9999) // e.g., 2006
+ month := parseUint(s[5:7], 1, 12) // e.g., 01
+ day := parseUint(s[8:10], 1, daysIn(time.Month(month), year)) // e.g., 02
+ hour := parseUint(s[11:13], 0, 23) // e.g., 15
+ min := parseUint(s[14:16], 0, 59) // e.g., 04
+ sec := parseUint(s[17:19], 0, 59) // e.g., 05
+ if !ok || !(s[4] == '-' && s[7] == '-' && s[10] == 'T' && s[13] == ':' && s[16] == ':') {
+ return time.Time{}, false
+ }
+ s = s[19:]
+
+ // Parse the fractional second.
+ var nsec int
+ if len(s) >= 2 && s[0] == '.' && isDigit(s, 1) {
+ n := 2
+ for ; n < len(s) && isDigit(s, n); n++ {
+ }
+ nsec, _, _ = parseNanoseconds(s, n)
+ s = s[n:]
+ }
+
+ // Parse the time zone.
+ loc := time.UTC
+ if len(s) != 1 || s[0] != 'Z' {
+ if len(s) != len("-07:00") {
+ return time.Time{}, false
+ }
+ hr := parseUint(s[1:3], 0, 23) // e.g., 07
+ mm := parseUint(s[4:6], 0, 59) // e.g., 00
+ if !ok || !((s[0] == '-' || s[0] == '+') && s[3] == ':') {
+ return time.Time{}, false
+ }
+ zoneOffsetSecs := (hr*60 + mm) * 60
+ if s[0] == '-' {
+ zoneOffsetSecs = -zoneOffsetSecs
+ }
+ loc = time.FixedZone("", zoneOffsetSecs)
+ }
+ t := time.Date(year, time.Month(month), day, hour, min, sec, nsec, loc)
+
+ return t, true
+}
+
+func isDigit(s string, i int) bool {
+ if len(s) <= i {
+ return false
+ }
+ c := s[i]
+ return '0' <= c && c <= '9'
+}
+
+func parseNanoseconds(value string, nbytes int) (ns int, rangeErrString string, err error) {
+ if value[0] != '.' && value[0] != ',' {
+ err = errBadTimestamp
+ return
+ }
+ if nbytes > 10 {
+ value = value[:10]
+ nbytes = 10
+ }
+ if ns, err = strconv.Atoi(value[1:nbytes]); err != nil {
+ return
+ }
+ if ns < 0 {
+ rangeErrString = "fractional second"
+ return
+ }
+ // We need nanoseconds, which means scaling by the number
+ // of missing digits in the format, maximum length 10.
+ scaleDigits := 10 - nbytes
+ for i := 0; i < scaleDigits; i++ {
+ ns *= 10
+ }
+ return
+}
+
+// These are internal errors used by the date parsing code and are not ever
+// returned by public functions.
+var errBadTimestamp = errors.New("bad value for field")
+
+// daysBefore[m] counts the number of days in a non-leap year
+// before month m begins. There is an entry for m=12, counting
+// the number of days before January of next year (365).
+var daysBefore = [...]int32{
+ 0,
+ 31,
+ 31 + 28,
+ 31 + 28 + 31,
+ 31 + 28 + 31 + 30,
+ 31 + 28 + 31 + 30 + 31,
+ 31 + 28 + 31 + 30 + 31 + 30,
+ 31 + 28 + 31 + 30 + 31 + 30 + 31,
+ 31 + 28 + 31 + 30 + 31 + 30 + 31 + 31,
+ 31 + 28 + 31 + 30 + 31 + 30 + 31 + 31 + 30,
+ 31 + 28 + 31 + 30 + 31 + 30 + 31 + 31 + 30 + 31,
+ 31 + 28 + 31 + 30 + 31 + 30 + 31 + 31 + 30 + 31 + 30,
+ 31 + 28 + 31 + 30 + 31 + 30 + 31 + 31 + 30 + 31 + 30 + 31,
+}
+
+func daysIn(m time.Month, year int) int {
+ if m == time.February && isLeap(year) {
+ return 29
+ }
+ return int(daysBefore[m] - daysBefore[m-1])
+}
+
+func isLeap(year int) bool {
+ return year%4 == 0 && (year%100 != 0 || year%400 == 0)
+}
diff --git a/cty/function/stdlib/datetime_test.go b/cty/function/stdlib/datetime_test.go
index 7432f82..9324f14 100644
--- a/cty/function/stdlib/datetime_test.go
+++ b/cty/function/stdlib/datetime_test.go
@@ -188,9 +188,7 @@ func TestFormatDate(t *testing.T) {
},
{
cty.StringVal("2017-01-02T26:00:00Z"),
- // This one generates an odd message due to an apparent quirk in
- // the Go time parser. Ideally it would use "26" as the errant string.
- `not a valid RFC3339 timestamp: cannot use ":00:00Z" as hour`,
+ `not a valid RFC3339 timestamp: hour must be between 0 and 23 inclusive`,
},
{
cty.StringVal("2017-13-02T00:00:00Z"),
@@ -212,6 +210,30 @@ func TestFormatDate(t *testing.T) {
// producing a confusing error message.
`not a valid RFC3339 timestamp: cannot use "-02T00:00:00Z" as year`,
},
+ {
+ cty.StringVal(`2000-01-01T1:12:34Z`),
+ `not a valid RFC3339 timestamp: hour must have exactly two digits`,
+ },
+ {
+ cty.StringVal(`2000-01-01T01:1:34Z`),
+ `not a valid RFC3339 timestamp: minute must have exactly two digits`,
+ },
+ {
+ cty.StringVal(`2000-01-01T01:01:1Z`),
+ `not a valid RFC3339 timestamp: cannot use "1Z" as second`,
+ },
+ {
+ cty.StringVal(`2000-01-01T00:00:00,000Z`),
+ `not a valid RFC3339 timestamp: cannot use "," as timestamp segment`,
+ },
+ {
+ cty.StringVal(`2000-01-01T00:00:00+24:00`),
+ `not a valid RFC3339 timestamp: cannot use "+24:00" as UTC offset`,
+ },
+ {
+ cty.StringVal(`2000-01-01T00:00:00+00:60`),
+ `not a valid RFC3339 timestamp: cannot use "+00:60" as UTC offset`,
+ },
}
for _, test := range parseErrTests {
t.Run(fmt.Sprintf("%s parse error", test.Timestamp.AsString()), func(t *testing.T) {
@@ -226,4 +248,62 @@ func TestFormatDate(t *testing.T) {
}
})
}
+
+ parseSuccessTests := []struct {
+ input string
+ wantRFC3339 string
+ wantRFC850 string
+ }{
+ {
+ "2022-03-01T00:23:45Z",
+ "2022-03-01T00:23:45Z",
+ "Tuesday, 01-Mar-22 00:23:45 UTC",
+ },
+ {
+ "2022-03-01T00:23:45+00:00",
+ "2022-03-01T00:23:45Z",
+ "Tuesday, 01-Mar-22 00:23:45 UTC",
+ },
+ {
+ "2022-03-01T00:23:45+01:00",
+ "2022-03-01T00:23:45+01:00",
+ "Tuesday, 01-Mar-22 00:23:45 +0100",
+ },
+ {
+ "2022-03-01T00:23:45-01:00",
+ "2022-03-01T00:23:45-01:00",
+ "Tuesday, 01-Mar-22 00:23:45 -0100",
+ },
+ {
+ "1900-01-01T00:00:00Z",
+ "1900-01-01T00:00:00Z",
+ "Monday, 01-Jan-00 00:00:00 UTC",
+ },
+ }
+ const rfc3339Format = "YYYY-MM-DD'T'hh:mm:ssZ"
+ const rfc850Format = "EEEE, DD-MMM-YY hh:mm:ss ZZZ"
+ for _, test := range parseSuccessTests {
+ t.Run(fmt.Sprintf("%s parse success", test.input), func(t *testing.T) {
+ t.Run("RFC3339", func(t *testing.T) {
+ got, err := FormatDate(cty.StringVal(rfc3339Format), cty.StringVal(test.input))
+ if err != nil {
+ t.Fatalf("unexpected error: %s", err)
+ }
+
+ if got.AsString() != test.wantRFC3339 {
+ t.Fatalf("wrong result\ngot: %s\nwant: %s", got.AsString(), test.wantRFC3339)
+ }
+ })
+ t.Run("RFC850", func(t *testing.T) {
+ got, err := FormatDate(cty.StringVal(rfc850Format), cty.StringVal(test.input))
+ if err != nil {
+ t.Fatalf("unexpected error: %s", err)
+ }
+
+ if got.AsString() != test.wantRFC850 {
+ t.Fatalf("wrong result\ngot: %s\nwant: %s", got.AsString(), test.wantRFC850)
+ }
+ })
+ })
+ }
}
diff --git a/cty/function/stdlib/format.go b/cty/function/stdlib/format.go
index ca163a8..d04a5ee 100644
--- a/cty/function/stdlib/format.go
+++ b/cty/function/stdlib/format.go
@@ -26,17 +26,27 @@ var FormatFunc = function.New(&function.Spec{
},
},
VarParam: &function.Parameter{
- Name: "args",
- Type: cty.DynamicPseudoType,
- AllowNull: true,
+ Name: "args",
+ Type: cty.DynamicPseudoType,
+ AllowNull: true,
+ AllowUnknown: true,
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
for _, arg := range args[1:] {
if !arg.IsWhollyKnown() {
// We require all nested values to be known because the only
// thing we can do for a collection/structural type is print
// it as JSON and that requires it to be wholly known.
+ // However, we might be able to refine the result with a
+ // known prefix, if there are literal characters before the
+ // first formatting verb.
+ f := args[0].AsString()
+ if idx := strings.IndexByte(f, '%'); idx > 0 {
+ prefix := f[:idx]
+ return cty.UnknownVal(cty.String).Refine().StringPrefix(prefix).NewValue(), nil
+ }
return cty.UnknownVal(cty.String), nil
}
}
@@ -59,7 +69,8 @@ var FormatListFunc = function.New(&function.Spec{
AllowNull: true,
AllowUnknown: true,
},
- Type: function.StaticReturnType(cty.List(cty.String)),
+ Type: function.StaticReturnType(cty.List(cty.String)),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
fmtVal := args[0]
args = args[1:]
@@ -164,7 +175,7 @@ var FormatListFunc = function.New(&function.Spec{
// We require all nested values to be known because the only
// thing we can do for a collection/structural type is print
// it as JSON and that requires it to be wholly known.
- ret = append(ret, cty.UnknownVal(cty.String))
+ ret = append(ret, cty.UnknownVal(cty.String).RefineNotNull())
continue Results
}
}
diff --git a/cty/function/stdlib/format_test.go b/cty/function/stdlib/format_test.go
index aad73a8..95c9dcc 100644
--- a/cty/function/stdlib/format_test.go
+++ b/cty/function/stdlib/format_test.go
@@ -99,7 +99,7 @@ func TestFormat(t *testing.T) {
[]cty.Value{cty.TupleVal([]cty.Value{
cty.UnknownVal(cty.String),
})},
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).Refine().NotNull().StringPrefixFull("tuple with unknown ").NewValue(),
``,
},
{
@@ -445,7 +445,7 @@ func TestFormat(t *testing.T) {
{
cty.UnknownVal(cty.String),
[]cty.Value{cty.True},
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(),
``,
},
{
@@ -457,13 +457,28 @@ func TestFormat(t *testing.T) {
{
cty.StringVal("Hello, %s!"),
[]cty.Value{cty.UnknownVal(cty.String)},
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).Refine().NotNull().StringPrefixFull("Hello, ").NewValue(),
+ ``,
+ },
+ {
+ cty.StringVal("Hello%s"),
+ []cty.Value{cty.UnknownVal(cty.String)},
+ // We lose the trailing "o" in the prefix here because the unknown
+ // value could potentially start with a combining diacritic, which
+ // would therefore combine into a different character.
+ cty.UnknownVal(cty.String).Refine().NotNull().StringPrefixFull("Hell").NewValue(),
``,
},
{
cty.StringVal("Hello, %[2]s!"),
[]cty.Value{cty.UnknownVal(cty.String), cty.StringVal("Ermintrude")},
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).Refine().NotNull().StringPrefixFull("Hello, ").NewValue(),
+ ``,
+ },
+ {
+ cty.StringVal("%s!"),
+ []cty.Value{cty.UnknownVal(cty.String)},
+ cty.UnknownVal(cty.String).RefineNotNull(),
``,
},
@@ -752,7 +767,7 @@ func TestFormatList(t *testing.T) {
[]cty.Value{
cty.True,
},
- cty.UnknownVal(cty.List(cty.String)),
+ cty.UnknownVal(cty.List(cty.String)).RefineNotNull(),
``,
},
15: {
@@ -761,7 +776,7 @@ func TestFormatList(t *testing.T) {
cty.UnknownVal(cty.String),
},
cty.ListVal([]cty.Value{
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(),
}),
``,
},
@@ -780,7 +795,7 @@ func TestFormatList(t *testing.T) {
[]cty.Value{
cty.UnknownVal(cty.List(cty.String)),
},
- cty.UnknownVal(cty.List(cty.String)),
+ cty.UnknownVal(cty.List(cty.String)).RefineNotNull(),
``,
},
18: {
@@ -794,7 +809,7 @@ func TestFormatList(t *testing.T) {
},
cty.ListVal([]cty.Value{
cty.StringVal(`["hello"]`),
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(),
cty.StringVal(`["world"]`),
}),
``,
@@ -804,7 +819,7 @@ func TestFormatList(t *testing.T) {
[]cty.Value{
cty.UnknownVal(cty.Tuple([]cty.Type{cty.String})),
},
- cty.UnknownVal(cty.List(cty.String)),
+ cty.UnknownVal(cty.List(cty.String)).RefineNotNull(),
``,
},
20: {
@@ -813,7 +828,7 @@ func TestFormatList(t *testing.T) {
cty.UnknownVal(cty.Tuple([]cty.Type{cty.String})),
cty.UnknownVal(cty.Tuple([]cty.Type{cty.String, cty.String})),
},
- cty.UnknownVal(cty.List(cty.String)),
+ cty.UnknownVal(cty.List(cty.String)).RefineNotNull(),
`argument 2 has length 2, which is inconsistent with argument 1 of length 1`,
},
21: {
@@ -822,7 +837,7 @@ func TestFormatList(t *testing.T) {
cty.ListVal([]cty.Value{cty.StringVal("hi")}),
cty.UnknownVal(cty.Tuple([]cty.Type{cty.String, cty.String})),
},
- cty.UnknownVal(cty.List(cty.String)),
+ cty.UnknownVal(cty.List(cty.String)).RefineNotNull(),
`argument 2 has length 2, which is inconsistent with argument 1 of length 1`,
},
22: {
@@ -833,7 +848,7 @@ func TestFormatList(t *testing.T) {
cty.UnknownVal(cty.String),
}),
},
- cty.UnknownVal(cty.List(cty.String)),
+ cty.UnknownVal(cty.List(cty.String)).RefineNotNull(),
``,
},
23: {
diff --git a/cty/function/stdlib/general.go b/cty/function/stdlib/general.go
index 4f70fff..627b55a 100644
--- a/cty/function/stdlib/general.go
+++ b/cty/function/stdlib/general.go
@@ -26,7 +26,8 @@ var EqualFunc = function.New(&function.Spec{
AllowNull: true,
},
},
- Type: function.StaticReturnType(cty.Bool),
+ Type: function.StaticReturnType(cty.Bool),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
return args[0].Equals(args[1]), nil
},
@@ -50,7 +51,8 @@ var NotEqualFunc = function.New(&function.Spec{
AllowNull: true,
},
},
- Type: function.StaticReturnType(cty.Bool),
+ Type: function.StaticReturnType(cty.Bool),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
return args[0].Equals(args[1]).Not(), nil
},
@@ -77,6 +79,7 @@ var CoalesceFunc = function.New(&function.Spec{
}
return retType, nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
for _, argVal := range args {
if !argVal.IsKnown() {
@@ -92,6 +95,10 @@ var CoalesceFunc = function.New(&function.Spec{
},
})
+func refineNonNull(b *cty.RefinementBuilder) *cty.RefinementBuilder {
+ return b.NotNull()
+}
+
// Equal determines whether the two given values are equal, returning a
// bool value.
func Equal(a cty.Value, b cty.Value) (cty.Value, error) {
diff --git a/cty/function/stdlib/general_test.go b/cty/function/stdlib/general_test.go
index e930058..d95fd21 100644
--- a/cty/function/stdlib/general_test.go
+++ b/cty/function/stdlib/general_test.go
@@ -36,22 +36,22 @@ func TestEqual(t *testing.T) {
{
cty.NumberIntVal(1),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.UnknownVal(cty.Number),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.NumberIntVal(1),
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
}
@@ -97,15 +97,15 @@ func TestCoalesce(t *testing.T) {
},
{
[]cty.Value{cty.UnknownVal(cty.Bool), cty.True},
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
[]cty.Value{cty.UnknownVal(cty.Bool), cty.StringVal("hello")},
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(),
},
{
[]cty.Value{cty.DynamicVal, cty.True},
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
[]cty.Value{cty.DynamicVal},
diff --git a/cty/function/stdlib/json.go b/cty/function/stdlib/json.go
index 63dd320..6559776 100644
--- a/cty/function/stdlib/json.go
+++ b/cty/function/stdlib/json.go
@@ -1,6 +1,10 @@
package stdlib
import (
+ "bytes"
+ "strings"
+ "unicode/utf8"
+
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/function"
"github.com/zclconf/go-cty/cty/json"
@@ -12,18 +16,40 @@ var JSONEncodeFunc = function.New(&function.Spec{
{
Name: "val",
Type: cty.DynamicPseudoType,
+ AllowUnknown: true,
AllowDynamicType: true,
AllowNull: true,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
val := args[0]
if !val.IsWhollyKnown() {
// We can't serialize unknowns, so if the value is unknown or
// contains any _nested_ unknowns then our result must be
- // unknown.
- return cty.UnknownVal(retType), nil
+ // unknown. However, we might still be able to at least constrain
+ // the prefix of our string so that downstreams can sniff for
+ // whether it's valid JSON and what result types it could have.
+
+ valRng := val.Range()
+ if valRng.CouldBeNull() {
+ // If null is possible then we can't constrain the result
+ // beyond the type constraint, because the very first character
+ // of the string is what distinguishes a null.
+ return cty.UnknownVal(retType), nil
+ }
+ b := cty.UnknownVal(retType).Refine()
+ ty := valRng.TypeConstraint()
+ switch {
+ case ty == cty.String:
+ b = b.StringPrefixFull(`"`)
+ case ty.IsObjectType() || ty.IsMapType():
+ b = b.StringPrefixFull("{")
+ case ty.IsTupleType() || ty.IsListType() || ty.IsSetType():
+ b = b.StringPrefixFull("[")
+ }
+ return b.NewValue(), nil
}
if val.IsNull() {
@@ -35,6 +61,11 @@ var JSONEncodeFunc = function.New(&function.Spec{
return cty.NilVal, err
}
+ // json.Marshal should already produce a trimmed string, but we'll
+ // make sure it always is because our unknown value refinements above
+ // assume there will be no leading whitespace before the value.
+ buf = bytes.TrimSpace(buf)
+
return cty.StringVal(string(buf)), nil
},
})
@@ -50,6 +81,42 @@ var JSONDecodeFunc = function.New(&function.Spec{
Type: func(args []cty.Value) (cty.Type, error) {
str := args[0]
if !str.IsKnown() {
+ // If the string isn't known then we can't fully parse it, but
+ // if the value has been refined with a prefix then we may at
+ // least be able to reject obviously-invalid syntax and maybe
+ // even predict the result type. It's safe to return a specific
+ // result type only if parsing a full document with this prefix
+ // would return exactly that type or fail with a syntax error.
+ rng := str.Range()
+ if prefix := strings.TrimSpace(rng.StringPrefix()); prefix != "" {
+ // If we know at least one character then it should be one
+ // of the few characters that can introduce a JSON value.
+ switch r, _ := utf8.DecodeRuneInString(prefix); r {
+ case '{', '[':
+ // These can start object values and array values
+ // respectively, but we can't actually form a full
+ // object type constraint or tuple type constraint
+ // without knowing all of the attributes, so we
+ // will still return DynamicPseudoType in this case.
+ case '"':
+ // This means that the result will either be a string
+ // or parsing will fail.
+ return cty.String, nil
+ case 't', 'f':
+ // Must either be a boolean value or a syntax error.
+ return cty.Bool, nil
+ case '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '.':
+ // These characters would all start the "number" production.
+ return cty.Number, nil
+ case 'n':
+ // n is valid to begin the keyword "null" but that doesn't
+ // give us any extra type information.
+ default:
+ // No other characters are valid as the beginning of a
+ // JSON value, so we can safely return an early error.
+ return cty.NilType, function.NewArgErrorf(0, "a JSON document cannot begin with the character %q", r)
+ }
+ }
return cty.DynamicPseudoType, nil
}
diff --git a/cty/function/stdlib/json_test.go b/cty/function/stdlib/json_test.go
index 5c1348f..3d182a9 100644
--- a/cty/function/stdlib/json_test.go
+++ b/cty/function/stdlib/json_test.go
@@ -42,15 +42,35 @@ func TestJSONEncode(t *testing.T) {
},
{
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(),
},
{
cty.ObjectVal(map[string]cty.Value{"dunno": cty.UnknownVal(cty.Bool), "false": cty.False}),
+ cty.UnknownVal(cty.String).Refine().NotNull().StringPrefixFull("{").NewValue(),
+ },
+ {
+ cty.ListVal([]cty.Value{cty.UnknownVal(cty.String)}),
+ cty.UnknownVal(cty.String).Refine().NotNull().StringPrefixFull("[").NewValue(),
+ },
+ {
cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(), // Can't refine the prefix because the input might be null
+ },
+ {
+ cty.UnknownVal(cty.String).RefineNotNull(),
+ cty.UnknownVal(cty.String).Refine().NotNull().StringPrefixFull(`"`).NewValue(),
+ },
+ {
+ cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.String).RefineNotNull(),
+ },
+ {
+ cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.String).RefineNotNull(),
},
{
cty.DynamicVal,
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(),
},
{
cty.NullVal(cty.String),
@@ -106,6 +126,38 @@ func TestJSONDecode(t *testing.T) {
cty.UnknownVal(cty.String),
cty.DynamicVal, // need to know the value to determine the type
},
+ {
+ cty.UnknownVal(cty.String).Refine().StringPrefixFull("1").NewValue(),
+ cty.UnknownVal(cty.Number), // deduced from refinement
+ },
+ {
+ cty.UnknownVal(cty.String).Refine().StringPrefixFull("-").NewValue(),
+ cty.UnknownVal(cty.Number), // deduced from refinement
+ },
+ {
+ cty.UnknownVal(cty.String).Refine().StringPrefixFull(".").NewValue(),
+ cty.UnknownVal(cty.Number), // deduced from refinement
+ },
+ {
+ cty.UnknownVal(cty.String).Refine().StringPrefixFull("t").NewValue(),
+ cty.UnknownVal(cty.Bool), // deduced from refinement
+ },
+ {
+ cty.UnknownVal(cty.String).Refine().StringPrefixFull("f").NewValue(),
+ cty.UnknownVal(cty.Bool), // deduced from refinement
+ },
+ {
+ cty.UnknownVal(cty.String).Refine().StringPrefixFull(`"blurt`).NewValue(),
+ cty.UnknownVal(cty.String), // deduced from refinement
+ },
+ {
+ cty.UnknownVal(cty.String).Refine().StringPrefixFull(`{`).NewValue(),
+ cty.DynamicVal, // can't deduce the result type, but potentially valid syntax
+ },
+ {
+ cty.UnknownVal(cty.String).Refine().StringPrefixFull(`[`).NewValue(),
+ cty.DynamicVal, // can't deduce the result type, but potentially valid syntax
+ },
{
cty.DynamicVal,
cty.DynamicVal,
@@ -129,4 +181,34 @@ func TestJSONDecode(t *testing.T) {
}
})
}
+
+ errorTests := []struct {
+ Input cty.Value
+ WantError string
+ }{
+ {
+ cty.StringVal("aaaa"),
+ `invalid character 'a' looking for beginning of value`,
+ },
+ {
+ cty.StringVal("nope"),
+ `invalid character 'o' in literal null (expecting 'u')`, // (the 'n' looked like the beginning of 'null')
+ },
+ {
+ cty.UnknownVal(cty.String).Refine().StringPrefixFull(`a`).NewValue(),
+ `a JSON document cannot begin with the character 'a'`, // error deduced from refinement, despite full value being unknown
+ },
+ }
+ for _, test := range errorTests {
+ t.Run(fmt.Sprintf("JSONDecode(%#v)", test.Input), func(t *testing.T) {
+ _, err := JSONDecode(test.Input)
+ if err == nil {
+ t.Fatal("unexpected success")
+ }
+
+ if got, want := err.Error(), test.WantError; got != want {
+ t.Errorf("wrong error\ngot: %s\nwant: %s", got, want)
+ }
+ })
+ }
}
diff --git a/cty/function/stdlib/number.go b/cty/function/stdlib/number.go
index ce73751..73ef32f 100644
--- a/cty/function/stdlib/number.go
+++ b/cty/function/stdlib/number.go
@@ -20,7 +20,8 @@ var AbsoluteFunc = function.New(&function.Spec{
AllowMarked: true,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
return args[0].Absolute(), nil
},
@@ -40,7 +41,8 @@ var AddFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
// big.Float.Add can panic if the input values are opposing infinities,
// so we must catch that here in order to remain within
@@ -74,7 +76,8 @@ var SubtractFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
// big.Float.Sub can panic if the input values are infinities,
// so we must catch that here in order to remain within
@@ -108,7 +111,8 @@ var MultiplyFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
// big.Float.Mul can panic if the input values are both zero or both
// infinity, so we must catch that here in order to remain within
@@ -143,7 +147,8 @@ var DivideFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
// big.Float.Quo can panic if the input values are both zero or both
// infinity, so we must catch that here in order to remain within
@@ -178,7 +183,8 @@ var ModuloFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
// big.Float.Mul can panic if the input values are both zero or both
// infinity, so we must catch that here in order to remain within
@@ -205,17 +211,20 @@ var GreaterThanFunc = function.New(&function.Spec{
{
Name: "a",
Type: cty.Number,
+ AllowUnknown: true,
AllowDynamicType: true,
AllowMarked: true,
},
{
Name: "b",
Type: cty.Number,
+ AllowUnknown: true,
AllowDynamicType: true,
AllowMarked: true,
},
},
- Type: function.StaticReturnType(cty.Bool),
+ Type: function.StaticReturnType(cty.Bool),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
return args[0].GreaterThan(args[1]), nil
},
@@ -227,17 +236,20 @@ var GreaterThanOrEqualToFunc = function.New(&function.Spec{
{
Name: "a",
Type: cty.Number,
+ AllowUnknown: true,
AllowDynamicType: true,
AllowMarked: true,
},
{
Name: "b",
Type: cty.Number,
+ AllowUnknown: true,
AllowDynamicType: true,
AllowMarked: true,
},
},
- Type: function.StaticReturnType(cty.Bool),
+ Type: function.StaticReturnType(cty.Bool),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
return args[0].GreaterThanOrEqualTo(args[1]), nil
},
@@ -249,17 +261,20 @@ var LessThanFunc = function.New(&function.Spec{
{
Name: "a",
Type: cty.Number,
+ AllowUnknown: true,
AllowDynamicType: true,
AllowMarked: true,
},
{
Name: "b",
Type: cty.Number,
+ AllowUnknown: true,
AllowDynamicType: true,
AllowMarked: true,
},
},
- Type: function.StaticReturnType(cty.Bool),
+ Type: function.StaticReturnType(cty.Bool),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
return args[0].LessThan(args[1]), nil
},
@@ -271,17 +286,20 @@ var LessThanOrEqualToFunc = function.New(&function.Spec{
{
Name: "a",
Type: cty.Number,
+ AllowUnknown: true,
AllowDynamicType: true,
AllowMarked: true,
},
{
Name: "b",
Type: cty.Number,
+ AllowUnknown: true,
AllowDynamicType: true,
AllowMarked: true,
},
},
- Type: function.StaticReturnType(cty.Bool),
+ Type: function.StaticReturnType(cty.Bool),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
return args[0].LessThanOrEqualTo(args[1]), nil
},
@@ -297,7 +315,8 @@ var NegateFunc = function.New(&function.Spec{
AllowMarked: true,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
return args[0].Negate(), nil
},
@@ -311,7 +330,8 @@ var MinFunc = function.New(&function.Spec{
Type: cty.Number,
AllowDynamicType: true,
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
if len(args) == 0 {
return cty.NilVal, fmt.Errorf("must pass at least one number")
@@ -336,7 +356,8 @@ var MaxFunc = function.New(&function.Spec{
Type: cty.Number,
AllowDynamicType: true,
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
if len(args) == 0 {
return cty.NilVal, fmt.Errorf("must pass at least one number")
@@ -362,7 +383,8 @@ var IntFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
bf := args[0].AsBigFloat()
if bf.IsInt() {
@@ -384,7 +406,8 @@ var CeilFunc = function.New(&function.Spec{
Type: cty.Number,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
f := args[0].AsBigFloat()
@@ -414,7 +437,8 @@ var FloorFunc = function.New(&function.Spec{
Type: cty.Number,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
f := args[0].AsBigFloat()
@@ -447,7 +471,8 @@ var LogFunc = function.New(&function.Spec{
Type: cty.Number,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
var num float64
if err := gocty.FromCtyValue(args[0], &num); err != nil {
@@ -476,7 +501,8 @@ var PowFunc = function.New(&function.Spec{
Type: cty.Number,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
var num float64
if err := gocty.FromCtyValue(args[0], &num); err != nil {
@@ -502,7 +528,8 @@ var SignumFunc = function.New(&function.Spec{
Type: cty.Number,
},
},
- Type: function.StaticReturnType(cty.Number),
+ Type: function.StaticReturnType(cty.Number),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
var num int
if err := gocty.FromCtyValue(args[0], &num); err != nil {
@@ -539,6 +566,7 @@ var ParseIntFunc = function.New(&function.Spec{
}
return cty.Number, nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
var numstr string
diff --git a/cty/function/stdlib/number_test.go b/cty/function/stdlib/number_test.go
index 1286e14..bb12821 100644
--- a/cty/function/stdlib/number_test.go
+++ b/cty/function/stdlib/number_test.go
@@ -36,11 +36,11 @@ func TestAbsolute(t *testing.T) {
},
{
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
}
@@ -73,22 +73,22 @@ func TestAdd(t *testing.T) {
{
cty.NumberIntVal(1),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.UnknownVal(cty.Number),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.NumberIntVal(1),
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
}
@@ -121,22 +121,22 @@ func TestSubtract(t *testing.T) {
{
cty.NumberIntVal(1),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.UnknownVal(cty.Number),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.NumberIntVal(1),
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
}
@@ -169,22 +169,22 @@ func TestMultiply(t *testing.T) {
{
cty.NumberIntVal(1),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.UnknownVal(cty.Number),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.NumberIntVal(1),
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
}
@@ -237,22 +237,22 @@ func TestDivide(t *testing.T) {
{
cty.NumberIntVal(1),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.UnknownVal(cty.Number),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.NumberIntVal(1),
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
}
@@ -305,22 +305,22 @@ func TestModulo(t *testing.T) {
{
cty.NumberIntVal(1),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.UnknownVal(cty.Number),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.NumberIntVal(1),
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
}
@@ -350,11 +350,11 @@ func TestNegate(t *testing.T) {
},
{
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
}
@@ -397,22 +397,27 @@ func TestLessThan(t *testing.T) {
{
cty.NumberIntVal(1),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
+ },
+ {
+ cty.NumberIntVal(1),
+ cty.UnknownVal(cty.Number).Refine().NumberRangeLowerBound(cty.NumberIntVal(2), true).NewValue(),
+ cty.True, // deduced from refinement
},
{
cty.UnknownVal(cty.Number),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.NumberIntVal(1),
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
}
@@ -455,22 +460,22 @@ func TestLessThanOrEqualTo(t *testing.T) {
{
cty.NumberIntVal(1),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.UnknownVal(cty.Number),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.NumberIntVal(1),
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
}
@@ -513,22 +518,22 @@ func TestGreaterThan(t *testing.T) {
{
cty.NumberIntVal(1),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.UnknownVal(cty.Number),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.NumberIntVal(1),
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
}
@@ -571,22 +576,22 @@ func TestGreaterThanOrEqualTo(t *testing.T) {
{
cty.NumberIntVal(1),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.UnknownVal(cty.Number),
cty.UnknownVal(cty.Number),
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.NumberIntVal(1),
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
{
cty.DynamicVal,
cty.DynamicVal,
- cty.UnknownVal(cty.Bool),
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
},
}
@@ -640,11 +645,11 @@ func TestMin(t *testing.T) {
},
{
[]cty.Value{cty.PositiveInfinity, cty.UnknownVal(cty.Number)},
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
[]cty.Value{cty.PositiveInfinity, cty.DynamicVal},
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
[]cty.Value{cty.Zero.Mark(1), cty.NumberIntVal(1)},
@@ -702,11 +707,11 @@ func TestMax(t *testing.T) {
},
{
[]cty.Value{cty.PositiveInfinity, cty.UnknownVal(cty.Number)},
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
{
[]cty.Value{cty.PositiveInfinity, cty.DynamicVal},
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
},
}
@@ -1170,55 +1175,55 @@ func TestParseInt(t *testing.T) {
{
cty.StringVal("FF"),
cty.NumberIntVal(10),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
true,
},
{
cty.StringVal("00FF"),
cty.NumberIntVal(10),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
true,
},
{
cty.StringVal("-00FF"),
cty.NumberIntVal(10),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
true,
},
{
cty.NumberIntVal(2),
cty.NumberIntVal(10),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
true,
},
{
cty.StringVal("1"),
cty.NumberIntVal(63),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
true,
},
{
cty.StringVal("1"),
cty.NumberIntVal(-1),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
true,
},
{
cty.StringVal("1"),
cty.NumberIntVal(1),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
true,
},
{
cty.StringVal("1"),
cty.NumberIntVal(0),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
true,
},
{
cty.StringVal("1.2"),
cty.NumberIntVal(10),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).RefineNotNull(),
true,
},
}
diff --git a/cty/function/stdlib/regexp.go b/cty/function/stdlib/regexp.go
index ab4257b..2465444 100644
--- a/cty/function/stdlib/regexp.go
+++ b/cty/function/stdlib/regexp.go
@@ -33,6 +33,7 @@ var RegexFunc = function.New(&function.Spec{
}
return retTy, err
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
if retType == cty.DynamicPseudoType {
return cty.DynamicVal, nil
@@ -79,6 +80,7 @@ var RegexAllFunc = function.New(&function.Spec{
}
return cty.List(retTy), err
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
ety := retType.ElementType()
if ety == cty.DynamicPseudoType {
diff --git a/cty/function/stdlib/regexp_test.go b/cty/function/stdlib/regexp_test.go
index d2fcb59..6bba064 100644
--- a/cty/function/stdlib/regexp_test.go
+++ b/cty/function/stdlib/regexp_test.go
@@ -43,14 +43,14 @@ func TestRegex(t *testing.T) {
cty.UnknownVal(cty.Tuple([]cty.Type{
cty.String,
cty.String,
- })),
+ })).RefineNotNull(),
},
{
cty.StringVal("(?P<num>[0-9]*)"),
cty.UnknownVal(cty.String),
cty.UnknownVal(cty.Object(map[string]cty.Type{
"num": cty.String,
- })),
+ })).RefineNotNull(),
},
{
cty.UnknownVal(cty.String),
@@ -134,19 +134,19 @@ func TestRegexAll(t *testing.T) {
cty.UnknownVal(cty.List(cty.Tuple([]cty.Type{
cty.String,
cty.String,
- }))),
+ }))).RefineNotNull(),
},
{
cty.StringVal("(?P<num>[0-9]*)"),
cty.UnknownVal(cty.String),
cty.UnknownVal(cty.List(cty.Object(map[string]cty.Type{
"num": cty.String,
- }))),
+ }))).RefineNotNull(),
},
{
cty.UnknownVal(cty.String),
cty.StringVal("135abc456def"),
- cty.UnknownVal(cty.List(cty.DynamicPseudoType)),
+ cty.UnknownVal(cty.List(cty.DynamicPseudoType)).RefineNotNull(),
},
}
diff --git a/cty/function/stdlib/sequence.go b/cty/function/stdlib/sequence.go
index 6b2d97b..009949d 100644
--- a/cty/function/stdlib/sequence.go
+++ b/cty/function/stdlib/sequence.go
@@ -74,6 +74,7 @@ var ConcatFunc = function.New(&function.Spec{
}
return cty.Tuple(etys), nil
},
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
switch {
case retType.IsListType():
@@ -143,7 +144,8 @@ var RangeFunc = function.New(&function.Spec{
Name: "params",
Type: cty.Number,
},
- Type: function.StaticReturnType(cty.List(cty.Number)),
+ Type: function.StaticReturnType(cty.List(cty.Number)),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
var start, end, step cty.Value
switch len(args) {
diff --git a/cty/function/stdlib/set.go b/cty/function/stdlib/set.go
index 15f4c05..6da2291 100644
--- a/cty/function/stdlib/set.go
+++ b/cty/function/stdlib/set.go
@@ -23,7 +23,8 @@ var SetHasElementFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(cty.Bool),
+ Type: function.StaticReturnType(cty.Bool),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
return args[0].HasElement(args[1]), nil
},
@@ -43,7 +44,8 @@ var SetUnionFunc = function.New(&function.Spec{
Type: cty.Set(cty.DynamicPseudoType),
AllowDynamicType: true,
},
- Type: setOperationReturnType,
+ Type: setOperationReturnType,
+ RefineResult: refineNonNull,
Impl: setOperationImpl(func(s1, s2 cty.ValueSet) cty.ValueSet {
return s1.Union(s2)
}, true),
@@ -63,7 +65,8 @@ var SetIntersectionFunc = function.New(&function.Spec{
Type: cty.Set(cty.DynamicPseudoType),
AllowDynamicType: true,
},
- Type: setOperationReturnType,
+ Type: setOperationReturnType,
+ RefineResult: refineNonNull,
Impl: setOperationImpl(func(s1, s2 cty.ValueSet) cty.ValueSet {
return s1.Intersection(s2)
}, false),
@@ -83,7 +86,8 @@ var SetSubtractFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: setOperationReturnType,
+ Type: setOperationReturnType,
+ RefineResult: refineNonNull,
Impl: setOperationImpl(func(s1, s2 cty.ValueSet) cty.ValueSet {
return s1.Subtract(s2)
}, false),
@@ -103,7 +107,8 @@ var SetSymmetricDifferenceFunc = function.New(&function.Spec{
Type: cty.Set(cty.DynamicPseudoType),
AllowDynamicType: true,
},
- Type: setOperationReturnType,
+ Type: setOperationReturnType,
+ RefineResult: refineNonNull,
Impl: setOperationImpl(func(s1, s2 cty.ValueSet) cty.ValueSet {
return s1.SymmetricDifference(s2)
}, false),
diff --git a/cty/function/stdlib/set_test.go b/cty/function/stdlib/set_test.go
index 71c8a20..9dce278 100644
--- a/cty/function/stdlib/set_test.go
+++ b/cty/function/stdlib/set_test.go
@@ -81,7 +81,7 @@ func TestSetUnion(t *testing.T) {
cty.SetVal([]cty.Value{cty.StringVal("5")}),
cty.UnknownVal(cty.Set(cty.Number)),
},
- cty.UnknownVal(cty.Set(cty.String)),
+ cty.UnknownVal(cty.Set(cty.String)).RefineNotNull(),
},
{
[]cty.Value{
@@ -178,14 +178,14 @@ func TestSetIntersection(t *testing.T) {
cty.SetVal([]cty.Value{cty.StringVal("5")}),
cty.UnknownVal(cty.Set(cty.Number)),
},
- cty.UnknownVal(cty.Set(cty.String)),
+ cty.UnknownVal(cty.Set(cty.String)).RefineNotNull(),
},
{
[]cty.Value{
cty.SetVal([]cty.Value{cty.StringVal("5")}),
cty.SetVal([]cty.Value{cty.UnknownVal(cty.String)}),
},
- cty.UnknownVal(cty.Set(cty.String)),
+ cty.UnknownVal(cty.Set(cty.String)).RefineNotNull(),
},
}
@@ -257,12 +257,12 @@ func TestSetSubtract(t *testing.T) {
{
cty.SetVal([]cty.Value{cty.StringVal("5")}),
cty.UnknownVal(cty.Set(cty.Number)),
- cty.UnknownVal(cty.Set(cty.String)),
+ cty.UnknownVal(cty.Set(cty.String)).RefineNotNull(),
},
{
cty.SetVal([]cty.Value{cty.StringVal("5")}),
cty.SetVal([]cty.Value{cty.UnknownVal(cty.String)}),
- cty.UnknownVal(cty.Set(cty.String)),
+ cty.UnknownVal(cty.Set(cty.String)).RefineNotNull(),
},
}
@@ -334,12 +334,12 @@ func TestSetSymmetricDifference(t *testing.T) {
{
cty.SetVal([]cty.Value{cty.StringVal("5")}),
cty.UnknownVal(cty.Set(cty.Number)),
- cty.UnknownVal(cty.Set(cty.String)),
+ cty.UnknownVal(cty.Set(cty.String)).RefineNotNull(),
},
{
cty.SetVal([]cty.Value{cty.StringVal("5")}),
cty.SetVal([]cty.Value{cty.UnknownVal(cty.Number)}),
- cty.UnknownVal(cty.Set(cty.String)),
+ cty.UnknownVal(cty.Set(cty.String)).RefineNotNull(),
},
}
diff --git a/cty/function/stdlib/string.go b/cty/function/stdlib/string.go
index f340ef7..57ebce1 100644
--- a/cty/function/stdlib/string.go
+++ b/cty/function/stdlib/string.go
@@ -22,7 +22,8 @@ var UpperFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
in := args[0].AsString()
out := strings.ToUpper(in)
@@ -39,7 +40,8 @@ var LowerFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
in := args[0].AsString()
out := strings.ToLower(in)
@@ -56,7 +58,8 @@ var ReverseFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
in := []byte(args[0].AsString())
out := make([]byte, len(in))
@@ -81,25 +84,46 @@ var StrlenFunc = function.New(&function.Spec{
{
Name: "str",
Type: cty.String,
+ AllowUnknown: true,
AllowDynamicType: true,
},
},
Type: function.StaticReturnType(cty.Number),
+ RefineResult: func(b *cty.RefinementBuilder) *cty.RefinementBuilder {
+ // String length is never null and never negative.
+ // (We might refine the lower bound even more inside Impl.)
+ return b.NotNull().NumberRangeLowerBound(cty.NumberIntVal(0), true)
+ },
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
- in := args[0].AsString()
- l := 0
-
- inB := []byte(in)
- for i := 0; i < len(in); {
- d, _, _ := textseg.ScanGraphemeClusters(inB[i:], true)
- l++
- i += d
+ if !args[0].IsKnown() {
+ ret := cty.UnknownVal(cty.Number)
+ // We may be able to still return a constrained result based on the
+ // refined range of the unknown value.
+ inRng := args[0].Range()
+ if inRng.TypeConstraint() == cty.String {
+ prefixLen := int64(graphemeClusterCount(inRng.StringPrefix()))
+ ret = ret.Refine().NumberRangeLowerBound(cty.NumberIntVal(prefixLen), true).NewValue()
+ }
+ return ret, nil
}
+ in := args[0].AsString()
+ l := graphemeClusterCount(in)
return cty.NumberIntVal(int64(l)), nil
},
})
+func graphemeClusterCount(in string) int {
+ l := 0
+ inB := []byte(in)
+ for i := 0; i < len(in); {
+ d, _, _ := textseg.ScanGraphemeClusters(inB[i:], true)
+ l++
+ i += d
+ }
+ return l
+}
+
var SubstrFunc = function.New(&function.Spec{
Description: "Extracts a substring from the given string.",
Params: []function.Parameter{
@@ -122,7 +146,8 @@ var SubstrFunc = function.New(&function.Spec{
AllowDynamicType: true,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
in := []byte(args[0].AsString())
var offset, length int
@@ -218,7 +243,8 @@ var JoinFunc = function.New(&function.Spec{
Description: "One or more lists of strings to join.",
Type: cty.List(cty.String),
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
sep := args[0].AsString()
listVals := args[1:]
@@ -258,18 +284,29 @@ var SortFunc = function.New(&function.Spec{
Description: "Applies a lexicographic sort to the elements of the given list.",
Params: []function.Parameter{
{
- Name: "list",
- Type: cty.List(cty.String),
+ Name: "list",
+ Type: cty.List(cty.String),
+ AllowUnknown: true,
},
},
- Type: function.StaticReturnType(cty.List(cty.String)),
+ Type: function.StaticReturnType(cty.List(cty.String)),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
listVal := args[0]
if !listVal.IsWhollyKnown() {
// If some of the element values aren't known yet then we
- // can't yet predict the order of the result.
- return cty.UnknownVal(retType), nil
+ // can't yet predict the order of the result, but we can be
+ // sure that the length won't change.
+ ret := cty.UnknownVal(retType)
+ if listVal.Type().IsListType() {
+ rng := listVal.Range()
+ ret = ret.Refine().
+ CollectionLengthLowerBound(rng.LengthLowerBound()).
+ CollectionLengthUpperBound(rng.LengthUpperBound()).
+ NewValue()
+ }
+ return ret, nil
}
if listVal.LengthInt() == 0 { // Easy path
return listVal, nil
@@ -307,7 +344,8 @@ var SplitFunc = function.New(&function.Spec{
Type: cty.String,
},
},
- Type: function.StaticReturnType(cty.List(cty.String)),
+ Type: function.StaticReturnType(cty.List(cty.String)),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
sep := args[0].AsString()
str := args[1].AsString()
@@ -333,7 +371,8 @@ var ChompFunc = function.New(&function.Spec{
Type: cty.String,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
newlines := regexp.MustCompile(`(?:\r\n?|\n)*\z`)
return cty.StringVal(newlines.ReplaceAllString(args[0].AsString(), "")), nil
@@ -356,7 +395,8 @@ var IndentFunc = function.New(&function.Spec{
Type: cty.String,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
var spaces int
if err := gocty.FromCtyValue(args[0], &spaces); err != nil {
@@ -378,7 +418,8 @@ var TitleFunc = function.New(&function.Spec{
Type: cty.String,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
return cty.StringVal(strings.Title(args[0].AsString())), nil
},
@@ -394,7 +435,8 @@ var TrimSpaceFunc = function.New(&function.Spec{
Type: cty.String,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
return cty.StringVal(strings.TrimSpace(args[0].AsString())), nil
},
@@ -416,7 +458,8 @@ var TrimFunc = function.New(&function.Spec{
Type: cty.String,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
str := args[0].AsString()
cutset := args[1].AsString()
@@ -443,7 +486,8 @@ var TrimPrefixFunc = function.New(&function.Spec{
Type: cty.String,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
str := args[0].AsString()
prefix := args[1].AsString()
@@ -467,7 +511,8 @@ var TrimSuffixFunc = function.New(&function.Spec{
Type: cty.String,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
str := args[0].AsString()
cutset := args[1].AsString()
diff --git a/cty/function/stdlib/string_replace.go b/cty/function/stdlib/string_replace.go
index 573083b..25a821b 100644
--- a/cty/function/stdlib/string_replace.go
+++ b/cty/function/stdlib/string_replace.go
@@ -30,7 +30,8 @@ var ReplaceFunc = function.New(&function.Spec{
Type: cty.String,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
str := args[0].AsString()
substr := args[1].AsString()
@@ -59,7 +60,8 @@ var RegexReplaceFunc = function.New(&function.Spec{
Type: cty.String,
},
},
- Type: function.StaticReturnType(cty.String),
+ Type: function.StaticReturnType(cty.String),
+ RefineResult: refineNonNull,
Impl: func(args []cty.Value, retType cty.Type) (ret cty.Value, err error) {
str := args[0].AsString()
substr := args[1].AsString()
diff --git a/cty/function/stdlib/string_test.go b/cty/function/stdlib/string_test.go
index 05fe941..254a2e7 100644
--- a/cty/function/stdlib/string_test.go
+++ b/cty/function/stdlib/string_test.go
@@ -1,6 +1,7 @@
package stdlib
import (
+ "fmt"
"testing"
"github.com/zclconf/go-cty/cty"
@@ -49,11 +50,11 @@ func TestUpper(t *testing.T) {
},
{
cty.UnknownVal(cty.String),
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(),
},
{
cty.DynamicVal,
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(),
},
{
cty.StringVal("hello").Mark(1),
@@ -103,11 +104,11 @@ func TestLower(t *testing.T) {
},
{
cty.UnknownVal(cty.String),
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(),
},
{
cty.DynamicVal,
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(),
},
}
@@ -174,11 +175,11 @@ func TestReverse(t *testing.T) {
},
{
cty.UnknownVal(cty.String),
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(),
},
{
cty.DynamicVal,
- cty.UnknownVal(cty.String),
+ cty.UnknownVal(cty.String).RefineNotNull(),
},
}
@@ -245,11 +246,15 @@ func TestStrlen(t *testing.T) {
},
{
cty.UnknownVal(cty.String),
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).Refine().NotNull().NumberRangeLowerBound(cty.Zero, true).NewValue(),
+ },
+ {
+ cty.UnknownVal(cty.String).Refine().StringPrefix("wé́́é́́é́́-").NewValue(),
+ cty.UnknownVal(cty.Number).Refine().NotNull().NumberRangeLowerBound(cty.NumberIntVal(5), true).NewValue(),
},
{
cty.DynamicVal,
- cty.UnknownVal(cty.Number),
+ cty.UnknownVal(cty.Number).Refine().NotNull().NumberRangeLowerBound(cty.Zero, true).NewValue(),
},
}
@@ -473,3 +478,82 @@ func TestJoin(t *testing.T) {
})
}
}
+
+func TestSort(t *testing.T) {
+ tests := []struct {
+ Input cty.Value
+ Want cty.Value
+ WantErr string
+ }{
+ {
+ cty.ListValEmpty(cty.String),
+ cty.ListValEmpty(cty.String),
+ ``,
+ },
+ {
+ cty.ListVal([]cty.Value{cty.StringVal("a")}),
+ cty.ListVal([]cty.Value{cty.StringVal("a")}),
+ ``,
+ },
+ {
+ cty.ListVal([]cty.Value{cty.StringVal("b"), cty.StringVal("a")}),
+ cty.ListVal([]cty.Value{cty.StringVal("a"), cty.StringVal("b")}),
+ ``,
+ },
+ {
+ cty.ListVal([]cty.Value{cty.StringVal("b"), cty.StringVal("a"), cty.StringVal("c")}),
+ cty.ListVal([]cty.Value{cty.StringVal("a"), cty.StringVal("b"), cty.StringVal("c")}),
+ ``,
+ },
+ {
+ cty.UnknownVal(cty.List(cty.String)),
+ cty.UnknownVal(cty.List(cty.String)).RefineNotNull(),
+ ``,
+ },
+ {
+ // If the list contains any unknown values then we can still
+ // preserve the length of the list by generating a known list
+ // with unknown elements, because sort can never change the length.
+ cty.ListVal([]cty.Value{cty.StringVal("b"), cty.UnknownVal(cty.String)}),
+ cty.ListVal([]cty.Value{cty.UnknownVal(cty.String), cty.UnknownVal(cty.String)}),
+ ``,
+ },
+ {
+ // For a completely unknown list we can still preserve any
+ // refinements it had for its length, because sorting can never
+ // change the length.
+ cty.UnknownVal(cty.List(cty.String)).Refine().
+ CollectionLengthLowerBound(1).
+ CollectionLengthUpperBound(2).
+ NewValue(),
+ cty.UnknownVal(cty.List(cty.String)).Refine().
+ NotNull().
+ CollectionLengthLowerBound(1).
+ CollectionLengthUpperBound(2).
+ NewValue(),
+ ``,
+ },
+ }
+
+ for _, test := range tests {
+ t.Run(fmt.Sprintf("Sort(%#v)", test.Input), func(t *testing.T) {
+ got, err := Sort(test.Input)
+
+ if test.WantErr != "" {
+ errStr := fmt.Sprintf("%s", err)
+ if errStr != test.WantErr {
+ t.Errorf("wrong error\ngot: %s\nwant: %s", errStr, test.WantErr)
+ }
+ return
+ }
+
+ if err != nil {
+ t.Fatalf("unexpected error: %s", err.Error())
+ }
+
+ if !got.RawEquals(test.Want) {
+ t.Errorf("wrong result\ninput: %#v\ngot: %#v\nwant: %#v", test.Input, got, test.Want)
+ }
+ })
+ }
+}
diff --git a/cty/helper.go b/cty/helper.go
index 1b88e9f..c342f13 100644
--- a/cty/helper.go
+++ b/cty/helper.go
@@ -8,7 +8,7 @@ import (
// unknowns, for operations that short-circuit to return unknown in that case.
func anyUnknown(values ...Value) bool {
for _, val := range values {
- if val.v == unknown {
+ if _, unknown := val.v.(*unknownType); unknown {
return true
}
}
@@ -39,7 +39,7 @@ func typeCheck(required Type, ret Type, values ...Value) (shortCircuit *Value, e
)
}
- if val.v == unknown {
+ if _, unknown := val.v.(*unknownType); unknown {
hasUnknown = true
}
}
diff --git a/cty/marks.go b/cty/marks.go
index b889e73..e747503 100644
--- a/cty/marks.go
+++ b/cty/marks.go
@@ -190,6 +190,9 @@ func (val Value) HasSameMarks(other Value) bool {
// An application that never calls this method does not need to worry about
// handling marked values.
func (val Value) Mark(mark interface{}) Value {
+ if _, ok := mark.(ValueMarks); ok {
+ panic("cannot call Value.Mark with a ValueMarks value (use WithMarks instead)")
+ }
var newMarker marker
newMarker.realV = val.v
if mr, ok := val.v.(marker); ok {
diff --git a/cty/msgpack/dynamic.go b/cty/msgpack/dynamic.go
index 9a4e94c..95d9160 100644
--- a/cty/msgpack/dynamic.go
+++ b/cty/msgpack/dynamic.go
@@ -3,7 +3,7 @@ package msgpack
import (
"bytes"
- "github.com/vmihailenco/msgpack/v4"
+ "github.com/vmihailenco/msgpack/v5"
"github.com/zclconf/go-cty/cty"
)
diff --git a/cty/msgpack/marshal.go b/cty/msgpack/marshal.go
index 2c4da8b..27da762 100644
--- a/cty/msgpack/marshal.go
+++ b/cty/msgpack/marshal.go
@@ -5,7 +5,7 @@ import (
"math/big"
"sort"
- "github.com/vmihailenco/msgpack/v4"
+ "github.com/vmihailenco/msgpack/v5"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert"
)
@@ -31,7 +31,8 @@ func Marshal(val cty.Value, ty cty.Type) ([]byte, error) {
var path cty.Path
var buf bytes.Buffer
enc := msgpack.NewEncoder(&buf)
- enc.UseCompactEncoding(true)
+ enc.UseCompactInts(true)
+ enc.UseCompactFloats(true)
err := marshal(val, ty, path, enc)
if err != nil {
@@ -53,11 +54,7 @@ func marshal(val cty.Value, ty cty.Type, path cty.Path, enc *msgpack.Encoder) er
}
if !val.IsKnown() {
- err := enc.Encode(unknownVal)
- if err != nil {
- return path.NewError(err)
- }
- return nil
+ return marshalUnknownValue(val.Range(), path, enc)
}
if val.IsNull() {
err := enc.EncodeNil()
diff --git a/cty/msgpack/roundtrip_test.go b/cty/msgpack/roundtrip_test.go
index 23de4e6..d82f126 100644
--- a/cty/msgpack/roundtrip_test.go
+++ b/cty/msgpack/roundtrip_test.go
@@ -38,6 +38,18 @@ func TestRoundTrip(t *testing.T) {
cty.UnknownVal(cty.String),
cty.String,
},
+ {
+ cty.UnknownVal(cty.String).RefineNotNull(),
+ cty.String,
+ },
+ {
+ cty.UnknownVal(cty.String).Refine().StringPrefix("foo-").NewValue(),
+ cty.String,
+ },
+ {
+ cty.UnknownVal(cty.String).Refine().NotNull().StringPrefix("foo-").NewValue(),
+ cty.String,
+ },
{
cty.True,
@@ -55,6 +67,10 @@ func TestRoundTrip(t *testing.T) {
cty.UnknownVal(cty.Bool),
cty.Bool,
},
+ {
+ cty.UnknownVal(cty.Bool).RefineNotNull(),
+ cty.Bool,
+ },
{
cty.NumberIntVal(1),
@@ -80,6 +96,34 @@ func TestRoundTrip(t *testing.T) {
cty.NegativeInfinity,
cty.Number,
},
+ {
+ cty.UnknownVal(cty.Number),
+ cty.Number,
+ },
+ {
+ cty.UnknownVal(cty.Number).RefineNotNull(),
+ cty.Number,
+ },
+ {
+ cty.UnknownVal(cty.Number).Refine().NumberRangeLowerBound(cty.Zero, true).NewValue(),
+ cty.Number,
+ },
+ {
+ cty.UnknownVal(cty.Number).Refine().NumberRangeLowerBound(cty.Zero, false).NewValue(),
+ cty.Number,
+ },
+ {
+ cty.UnknownVal(cty.Number).Refine().NumberRangeUpperBound(cty.Zero, true).NewValue(),
+ cty.Number,
+ },
+ {
+ cty.UnknownVal(cty.Number).Refine().NumberRangeUpperBound(cty.Zero, false).NewValue(),
+ cty.Number,
+ },
+ {
+ cty.UnknownVal(cty.Number).Refine().NumberRangeInclusive(cty.Zero, cty.NumberIntVal(1)).NewValue(),
+ cty.Number,
+ },
{
cty.ListVal([]cty.Value{
@@ -107,6 +151,35 @@ func TestRoundTrip(t *testing.T) {
cty.ListValEmpty(cty.String),
cty.List(cty.String),
},
+ {
+ cty.UnknownVal(cty.List(cty.String)),
+ cty.List(cty.String),
+ },
+ {
+ cty.UnknownVal(cty.List(cty.String)).RefineNotNull(),
+ cty.List(cty.String),
+ },
+ {
+ cty.UnknownVal(cty.List(cty.String)).Refine().CollectionLengthLowerBound(1).NewValue(),
+ cty.List(cty.String),
+ },
+ {
+ cty.UnknownVal(cty.List(cty.String)).Refine().CollectionLengthUpperBound(1).NewValue(),
+ cty.List(cty.String),
+ },
+ {
+ cty.UnknownVal(cty.List(cty.String)).Refine().CollectionLengthLowerBound(1).CollectionLengthUpperBound(2).NewValue(),
+ cty.List(cty.String),
+ },
+ {
+ // NOTE: This refinement should collapse to a known 2-element list with unknown elements
+ cty.UnknownVal(cty.List(cty.String)).Refine().CollectionLengthLowerBound(2).CollectionLengthUpperBound(2).NewValue(),
+ cty.List(cty.String),
+ },
+ {
+ cty.UnknownVal(cty.List(cty.String)).Refine().CollectionLengthUpperBound(1).NotNull().NewValue(),
+ cty.List(cty.String),
+ },
{
cty.SetVal([]cty.Value{
diff --git a/cty/msgpack/type_implied.go b/cty/msgpack/type_implied.go
index a169f28..4840063 100644
--- a/cty/msgpack/type_implied.go
+++ b/cty/msgpack/type_implied.go
@@ -5,8 +5,8 @@ import (
"fmt"
"io"
- "github.com/vmihailenco/msgpack/v4"
- msgpackcodes "github.com/vmihailenco/msgpack/v4/codes"
+ "github.com/vmihailenco/msgpack/v5"
+ msgpackcodes "github.com/vmihailenco/msgpack/v5/msgpcode"
"github.com/zclconf/go-cty/cty"
)
diff --git a/cty/msgpack/unknown.go b/cty/msgpack/unknown.go
index 6507bc4..b189ae8 100644
--- a/cty/msgpack/unknown.go
+++ b/cty/msgpack/unknown.go
@@ -1,16 +1,313 @@
package msgpack
+import (
+ "bytes"
+ "io"
+ "math"
+ "unicode/utf8"
+
+ "github.com/vmihailenco/msgpack/v5"
+ "github.com/zclconf/go-cty/cty"
+)
+
type unknownType struct{}
var unknownVal = unknownType{}
// unknownValBytes is the raw bytes of the msgpack fixext1 value we
-// write to represent an unknown value. It's an extension value of
+// write to represent a totally unknown value. It's an extension value of
// type zero whose value is irrelevant. Since it's irrelevant, we
// set it to a single byte whose value is also zero, since that's
// the most compact possible representation.
+//
+// The representation of a refined unknown value is different. See
+// marshalUnknownValue for more details.
var unknownValBytes = []byte{0xd4, 0, 0}
func (uv unknownType) MarshalMsgpack() ([]byte, error) {
return unknownValBytes, nil
}
+
+const unknownWithRefinementsExt = 0x0c
+
+type unknownValRefinementKey int64
+
+const unknownValNullness unknownValRefinementKey = 1
+const unknownValStringPrefix unknownValRefinementKey = 2
+const unknownValNumberMin unknownValRefinementKey = 3
+const unknownValNumberMax unknownValRefinementKey = 4
+const unknownValLengthMin unknownValRefinementKey = 5
+const unknownValLengthMax unknownValRefinementKey = 6
+
+func marshalUnknownValue(rng cty.ValueRange, path cty.Path, enc *msgpack.Encoder) error {
+ if rng.TypeConstraint() == cty.DynamicPseudoType {
+ // cty.DynamicVal can never have refinements
+ err := enc.Encode(unknownVal)
+ if err != nil {
+ return path.NewError(err)
+ }
+ return nil
+ }
+
+ var refnBuf bytes.Buffer
+ refnEnc := msgpack.NewEncoder(&refnBuf)
+ mapLen := 0
+
+ if rng.DefinitelyNotNull() {
+ mapLen++
+ refnEnc.EncodeInt(int64(unknownValNullness))
+ refnEnc.EncodeBool(false)
+ }
+ switch {
+ case rng.TypeConstraint() == cty.Number:
+ lower, lowerInc := rng.NumberLowerBound()
+ upper, upperInc := rng.NumberUpperBound()
+ boundTy := cty.Tuple([]cty.Type{cty.Number, cty.Bool})
+ if lower.IsKnown() && lower != cty.NegativeInfinity {
+ mapLen++
+ refnEnc.EncodeInt(int64(unknownValNumberMin))
+ marshal(
+ cty.TupleVal([]cty.Value{lower, cty.BoolVal(lowerInc)}),
+ boundTy,
+ nil,
+ refnEnc,
+ )
+ }
+ if upper.IsKnown() && upper != cty.PositiveInfinity {
+ mapLen++
+ refnEnc.EncodeInt(int64(unknownValNumberMax))
+ marshal(
+ cty.TupleVal([]cty.Value{upper, cty.BoolVal(upperInc)}),
+ boundTy,
+ nil,
+ refnEnc,
+ )
+ }
+ case rng.TypeConstraint() == cty.String:
+ if prefix := rng.StringPrefix(); prefix != "" {
+ mapLen++
+ refnEnc.EncodeInt(int64(unknownValStringPrefix))
+ refnEnc.EncodeString(prefix)
+ }
+ case rng.TypeConstraint().IsCollectionType():
+ lower := rng.LengthLowerBound()
+ upper := rng.LengthUpperBound()
+ if lower != 0 {
+ mapLen++
+ refnEnc.EncodeInt(int64(unknownValLengthMin))
+ refnEnc.EncodeInt(int64(lower))
+ }
+ if upper != math.MaxInt {
+ mapLen++
+ refnEnc.EncodeInt(int64(unknownValLengthMax))
+ refnEnc.EncodeInt(int64(upper))
+ }
+ }
+
+ if mapLen == 0 {
+ // No refinements to encode, so we'll use the old compact representation.
+ err := enc.Encode(unknownVal)
+ if err != nil {
+ return path.NewError(err)
+ }
+ return nil
+ }
+
+ // If we have at least one refinement to encode then we'll use the new
+ // representation of unknown values where refinement information is in the
+ // extension payload.
+ var lenBuf bytes.Buffer
+ lenEnc := msgpack.NewEncoder(&lenBuf)
+ lenEnc.EncodeMapLen(mapLen)
+
+ err := enc.EncodeExtHeader(unknownWithRefinementsExt, lenBuf.Len()+refnBuf.Len())
+ if err != nil {
+ return path.NewErrorf("failed to write unknown value: %s", err)
+ }
+ _, err = enc.Writer().Write(lenBuf.Bytes())
+ if err != nil {
+ return path.NewErrorf("failed to write unknown value: %s", err)
+ }
+ _, err = enc.Writer().Write(refnBuf.Bytes())
+ if err != nil {
+ return path.NewErrorf("failed to write unknown value: %s", err)
+ }
+ return nil
+}
+
+func unmarshalUnknownValue(dec *msgpack.Decoder, ty cty.Type, path cty.Path) (cty.Value, error) {
+ // The next item in the stream should be a msgpack extension value,
+ // which might be zero-length for a totally unknown value, or it might
+ // contain a mapping describing some type-specific refinements.
+ typeCode, extLen, err := dec.DecodeExtHeader()
+ if err != nil {
+ return cty.DynamicVal, path.NewErrorf("extension code is required for unknown value")
+ }
+
+ if extLen <= 1 {
+ // Zero-length or one-length extension represents an unknown value with
+ // no refinements. (msgpack's serialization of a zero-length extension
+ // is one byte longer than a one-byte extension, so the encoder uses
+ // one nul byte as its "totally unknown" encoding.
+
+ if extLen > 0 {
+ // We need to skip the body, then.
+ body := make([]byte, extLen)
+ _, err = io.ReadAtLeast(dec.Buffered(), body, len(body))
+ if err != nil {
+ return cty.DynamicVal, path.NewErrorf("failed to read msgpack extension body: %s", err)
+ }
+ }
+ return cty.UnknownVal(ty), nil
+ }
+
+ if typeCode != unknownWithRefinementsExt {
+ // If there's a non-zero length then we require a specific type code
+ // as an additional signal that the body is intended to be a refinement map.
+ return cty.DynamicVal, path.NewErrorf("unsupported extension type 0x%02x with len %d", typeCode, extLen)
+ }
+
+ if extLen > 1024 {
+ // A refinement description greater than 1 kiB is unreasonable and
+ // might be an abusive attempt to allocate large amounts of memory
+ // in a system consuming this input.
+ return cty.DynamicVal, path.NewErrorf("oversize unknown value refinement")
+ }
+
+ // If we get here then typeCode == 0xc and we have a non-zero length.
+ // We expect to find a msgpack-encoded map in the payload which describes
+ // any refinements to add to the result.
+ body := make([]byte, extLen)
+ _, err = io.ReadAtLeast(dec.Buffered(), body, len(body))
+ if err != nil {
+ return cty.DynamicVal, path.NewErrorf("failed to read msgpack extension body: %s", err)
+ }
+
+ rfnDec := msgpack.NewDecoder(bytes.NewReader(body))
+ entryCount, err := rfnDec.DecodeMapLen()
+ if err != nil {
+ return cty.DynamicVal, path.NewErrorf("failed to decode msgpack extension body: not a map")
+ }
+
+ if ty == cty.DynamicPseudoType {
+ // We'll silently ignore all refinements for DynamicPseudoType for now,
+ // since we know that's invalid today but we might find a way to
+ // support it in the future and if so will want to introduce that
+ // in a backward-compatible way.
+ return cty.UnknownVal(ty), nil
+ }
+
+ builder := cty.UnknownVal(ty).Refine()
+ for i := 0; i < entryCount; i++ {
+ // Our refinement encoding format uses compact msgpack primitives to
+ // minimize the encoding size of refinements, which could otherwise
+ // add up to be quite large for a payload containing lots of unknown
+ // values. The keys are small integers to fit in the positive fixint
+ // encoding scheme. The values are encoded differently depending on
+ // the key but also aim for compactness.
+ // The smallest possible non-empty refinement map is three bytes:
+ // - one byte to encode that it's a one-element map
+ // - one byte to encode the key
+ // - at least one byte to encode the value associated with that key
+ // Encoders should avoid encoding zero-length maps and prefer to
+ // leave the payload zero bytes long in that case.
+
+ keyCode, err := rfnDec.DecodeInt64()
+ if err != nil {
+ return cty.DynamicVal, path.NewErrorf("failed to decode msgpack extension body: non-integer key in map")
+ }
+
+ // Exactly which keys are supported depends on the destination type.
+ // We'll reject keys that we know can't possibly apply to the given
+ // type, but we'll ignore keys we haven't seen before to allow for
+ // future expansion of the possible refinements.
+ // These keys all have intentionally-short names
+ switch keyCode := unknownValRefinementKey(keyCode); keyCode {
+ case unknownValNullness:
+ isNull, err := rfnDec.DecodeBool()
+ if err != nil {
+ return cty.DynamicVal, path.NewErrorf("failed to decode msgpack extension body: null refinement is not boolean")
+ }
+ // The presence of this key means we're refining the null-ness one
+ // way or another. If nullness is unknown then this key should not
+ // be present at all.
+ if isNull {
+ // it'd be weird to actually serialize a refinement like
+ // this because trying to apply this refinement in the first
+ // place should've collapsed into a known null value. But we'll
+ // allow it anyway just for complete encoding of the current
+ // refinement model.
+ builder = builder.Null()
+ } else {
+ builder = builder.NotNull()
+ }
+ case unknownValStringPrefix:
+ if ty != cty.String {
+ return cty.DynamicVal, path.NewErrorf("failed to decode msgpack extension body: string prefix refinement for non-string type")
+ }
+ prefixStr, err := rfnDec.DecodeString()
+ if err != nil {
+ return cty.DynamicVal, path.NewErrorf("failed to decode msgpack extension body: string prefix refinement is not string")
+ }
+ if !utf8.ValidString(prefixStr) {
+ return cty.DynamicVal, path.NewErrorf("failed to decode msgpack extension body: string prefix refinement is not valid UTF-8")
+ }
+ // We assume that the original creator of this value already took
+ // care of making sure the prefix is safe, so we don't need to
+ // constrain it any further.
+ builder = builder.StringPrefixFull(prefixStr)
+ case unknownValLengthMin, unknownValLengthMax:
+ if !ty.IsCollectionType() {
+ return cty.DynamicVal, path.NewErrorf("failed to decode msgpack extension body: length lower bound refinement for non-collection type")
+ }
+
+ bound, err := rfnDec.DecodeInt()
+ if err != nil {
+ return cty.DynamicVal, path.NewErrorf("failed to decode msgpack extension body: length bound refinement must be integer or [integer, bool] array")
+ }
+ switch keyCode {
+ case unknownValLengthMin:
+ builder = builder.CollectionLengthLowerBound(bound)
+ case unknownValLengthMax:
+ builder = builder.CollectionLengthUpperBound(bound)
+ default:
+ panic("unsupported keyCode") // should not get here
+ }
+ case unknownValNumberMin, unknownValNumberMax:
+ if ty != cty.Number {
+ return cty.DynamicVal, path.NewErrorf("failed to decode msgpack extension body: numeric bound refinement for non-number type")
+ }
+ // We want to support all of the same various number encodings we
+ // support for normal numbers, so here we'll cheat a bit and decode
+ // using our own value unmarshal function.
+ rawBound, err := unmarshal(rfnDec, cty.Tuple([]cty.Type{cty.Number, cty.Bool}), nil)
+ if err != nil || rawBound.IsNull() || !rawBound.IsKnown() {
+ return cty.DynamicVal, path.NewErrorf("failed to decode msgpack extension body: length bound refinement must be [number, bool] array")
+ }
+ boundVal := rawBound.Index(cty.Zero)
+ isIncVal := rawBound.Index(cty.NumberIntVal(1))
+ if boundVal.Type() != cty.Number || !boundVal.IsKnown() || boundVal.IsNull() {
+ return cty.DynamicVal, path.NewErrorf("failed to decode msgpack extension body: length bound refinement must be [number, bool] array")
+ }
+ if isIncVal.Type() != cty.Bool || !isIncVal.IsKnown() || isIncVal.IsNull() {
+ return cty.DynamicVal, path.NewErrorf("failed to decode msgpack extension body: length bound refinement must be [number, bool] array")
+ }
+ isInc := isIncVal.True()
+ switch keyCode {
+ case unknownValNumberMin:
+ builder = builder.NumberRangeLowerBound(boundVal, isInc)
+ case unknownValNumberMax:
+ builder = builder.NumberRangeUpperBound(boundVal, isInc)
+ default:
+ panic("unsupported keyCode") // should not get here
+ }
+ }
+ }
+
+ // NOTE: We intentionally ignore any trailing bytes after the extension
+ // map in case we want to pack something else in there later or in case
+ // a future version wants to use padding to optimize storage. Current
+ // encoders should not add any extra content there, though.
+
+ return builder.NewValue(), nil
+}
diff --git a/cty/msgpack/unmarshal.go b/cty/msgpack/unmarshal.go
index 1ea0b0a..08da1f7 100644
--- a/cty/msgpack/unmarshal.go
+++ b/cty/msgpack/unmarshal.go
@@ -3,8 +3,8 @@ package msgpack
import (
"bytes"
- "github.com/vmihailenco/msgpack/v4"
- msgpackCodes "github.com/vmihailenco/msgpack/v4/codes"
+ "github.com/vmihailenco/msgpack/v5"
+ msgpackCodes "github.com/vmihailenco/msgpack/v5/msgpcode"
"github.com/zclconf/go-cty/cty"
)
@@ -30,8 +30,7 @@ func unmarshal(dec *msgpack.Decoder, ty cty.Type, path cty.Path) (cty.Value, err
if msgpackCodes.IsExt(peek) {
// We just assume _all_ extensions are unknown values,
// since we don't have any other extensions.
- dec.Skip() // skip what we've peeked
- return cty.UnknownVal(ty), nil
+ return unmarshalUnknownValue(dec, ty, path)
}
if ty == cty.DynamicPseudoType {
return unmarshalDynamic(dec, path)
diff --git a/cty/unknown.go b/cty/unknown.go
index 83893c0..b3aefa4 100644
--- a/cty/unknown.go
+++ b/cty/unknown.go
@@ -3,11 +3,19 @@ package cty
// unknownType is the placeholder type used for the sigil value representing
// "Unknown", to make it unambigiously distinct from any other possible value.
type unknownType struct {
+ // refinement is an optional object which, if present, describes some
+ // additional constraints we know about the range of real values this
+ // unknown value could be a placeholder for.
+ refinement unknownValRefinement
}
-// unknown is a special value that can be used as the internal value of a
-// Value to create a placeholder for a value that isn't yet known.
-var unknown interface{} = &unknownType{}
+// totallyUnknown is the representation a a value we know nothing about at
+// all. Subsequent refinements of an unknown value will cause creation of
+// other values of unknownType that can represent additional constraints
+// on the unknown value, but all unknown values start as totally unknown
+// and we will also typically lose all unknown value refinements when
+// round-tripping through serialization formats.
+var totallyUnknown interface{} = &unknownType{}
// UnknownVal returns an Value that represents an unknown value of the given
// type. Unknown values can be used to represent a value that is
@@ -19,7 +27,7 @@ var unknown interface{} = &unknownType{}
func UnknownVal(t Type) Value {
return Value{
ty: t,
- v: unknown,
+ v: totallyUnknown,
}
}
@@ -80,6 +88,6 @@ func init() {
}
DynamicVal = Value{
ty: DynamicPseudoType,
- v: unknown,
+ v: totallyUnknown,
}
}
diff --git a/cty/unknown_refinement.go b/cty/unknown_refinement.go
new file mode 100644
index 0000000..d90bcbc
--- /dev/null
+++ b/cty/unknown_refinement.go
@@ -0,0 +1,747 @@
+package cty
+
+import (
+ "fmt"
+ "math"
+ "strings"
+
+ "github.com/zclconf/go-cty/cty/ctystrings"
+)
+
+// Refine creates a [RefinementBuilder] with which to annotate the reciever
+// with zero or more additional refinements that constrain the range of
+// the value.
+//
+// Calling methods on a RefinementBuilder for a known value essentially just
+// serves as assertions about the range of that value, leading to panics if
+// those assertions don't hold in practice. This is mainly supported just to
+// make programs that rely on refinements automatically self-check by using
+// the refinement codepath unconditionally on both placeholders and final
+// values for those placeholders. It's always a bug to refine the range of
+// an unknown value and then later substitute an exact value outside of the
+// refined range.
+//
+// Calling methods on a RefinementBuilder for an unknown value is perhaps
+// more useful because the newly-refined value will then be a placeholder for
+// a smaller range of values and so it may be possible for other operations
+// on the unknown value to return a known result despite the exact value not
+// yet being known.
+//
+// It is never valid to refine [DynamicVal], because that value is a
+// placeholder for a value about which we knkow absolutely nothing. A value
+// must at least have a known root type before it can support further
+// refinement.
+func (v Value) Refine() *RefinementBuilder {
+ v, marks := v.Unmark()
+ if unk, isUnk := v.v.(*unknownType); isUnk && unk.refinement != nil {
+ // We're refining a value that's already been refined before, so
+ // we'll start from a copy of its existing refinements.
+ wip := unk.refinement.copy()
+ return &RefinementBuilder{v, marks, wip}
+ }
+
+ ty := v.Type()
+ var wip unknownValRefinement
+ switch {
+ case ty == DynamicPseudoType && !v.IsKnown():
+ panic("cannot refine an unknown value of an unknown type")
+ case ty == String:
+ wip = &refinementString{}
+ case ty == Number:
+ wip = &refinementNumber{}
+ case ty.IsCollectionType():
+ wip = &refinementCollection{
+ // A collection can never have a negative length, so we'll
+ // start with that already constrained.
+ minLen: 0,
+ maxLen: math.MaxInt,
+ }
+ case ty == Bool || ty.IsObjectType() || ty.IsTupleType() || ty.IsCapsuleType():
+ // For other known types we'll just track nullability
+ wip = &refinementNullable{}
+ case ty == DynamicPseudoType && v.IsNull():
+ // It's okay in principle to refine a null value of unknown type,
+ // although all we can refine about it is that it's definitely null and
+ // so this is pretty pointless and only supported to avoid callers
+ // always needing to treat this situation as a special case to avoid
+ // panic.
+ wip = &refinementNullable{
+ isNull: tristateTrue,
+ }
+ default:
+ // we leave "wip" as nil for all other types, representing that
+ // they don't support refinements at all and so any call on the
+ // RefinementBuilder should fail.
+
+ // NOTE: We intentionally don't allow any refinements for
+ // cty.DynamicVal here, even though it could be nice in principle
+ // to at least track non-nullness for those, because it's historically
+ // been valid to directly compare values with cty.DynamicVal using
+ // the Go "==" operator and recording a refinement for an untyped
+ // unknown value would break existing code relying on that.
+ }
+
+ return &RefinementBuilder{v, marks, wip}
+}
+
+// RefineWith is a variant of Refine which uses callback functions instead of
+// the builder pattern.
+//
+// The result is equivalent to passing the return value of [Value.Refine] to the
+// first callback, and then continue passing the builder through any other
+// callbacks in turn, and then calling [RefinementBuilder.NewValue] on the
+// final result.
+//
+// The builder pattern approach of [Value.Refine] is more convenient for inline
+// annotation of refinements when constructing a value, but this alternative
+// approach may be more convenient when applying pre-defined collections of
+// refinements, or when refinements are defined separately from the values
+// they will apply to.
+//
+// Each refiner callback should return the same pointer that it was given,
+// typically after having mutated it using the [RefinementBuilder] methods.
+// It's invalid to return a different builder.
+func (v Value) RefineWith(refiners ...func(*RefinementBuilder) *RefinementBuilder) Value {
+ if len(refiners) == 0 {
+ return v
+ }
+ origBuilder := v.Refine()
+ builder := origBuilder
+ for _, refiner := range refiners {
+ builder = refiner(builder)
+ if builder != origBuilder {
+ panic("refiner callback returned a different builder")
+ }
+ }
+ return builder.NewValue()
+}
+
+// RefineNotNull is a shorthand for Value.Refine().NotNull().NewValue(), because
+// declaring that a unknown value isn't null is by far the most common use of
+// refinements.
+func (v Value) RefineNotNull() Value {
+ return v.Refine().NotNull().NewValue()
+}
+
+// RefinementBuilder is a supporting type for the [Value.Refine] method,
+// using the builder pattern to apply zero or more constraints before
+// constructing a new value with all of those constraints applied.
+//
+// Most of the methods of this type return the same reciever to allow
+// for method call chaining. End call chains with a call to
+// [RefinementBuilder.NewValue] to obtain the newly-refined value.
+type RefinementBuilder struct {
+ orig Value
+ marks ValueMarks
+ wip unknownValRefinement
+}
+
+func (b *RefinementBuilder) assertRefineable() {
+ if b.wip == nil {
+ panic(fmt.Sprintf("cannot refine a %#v value", b.orig.Type()))
+ }
+}
+
+// NotNull constrains the value as definitely not being null.
+//
+// NotNull is valid when refining values of the following types:
+// - number, boolean, and string values
+// - list, set, or map types of any element type
+// - values of object types
+// - values of collection types
+// - values of capsule types
+//
+// When refining any other type this function will panic.
+//
+// In particular note that it is not valid to constrain an untyped value
+// -- a value whose type is `cty.DynamicPseudoType` -- as being non-null.
+// An unknown value of an unknown type is always completely unconstrained.
+func (b *RefinementBuilder) NotNull() *RefinementBuilder {
+ b.assertRefineable()
+
+ if b.orig.IsKnown() && b.orig.IsNull() {
+ panic("refining null value as non-null")
+ }
+ if b.wip.null() == tristateTrue {
+ panic("refining null value as non-null")
+ }
+
+ b.wip.setNull(tristateFalse)
+
+ return b
+}
+
+// Null constrains the value as definitely null.
+//
+// Null is valid for the same types as [RefinementBuilder.NotNull].
+// When refining any other type this function will panic.
+//
+// Explicitly cnstraining a value to be null is strange because that suggests
+// that the caller does actually know the value -- there is only one null
+// value for each type constraint -- but this is here for symmetry with the
+// fact that a [ValueRange] can also represent that a value is definitely null.
+func (b *RefinementBuilder) Null() *RefinementBuilder {
+ b.assertRefineable()
+
+ if b.orig.IsKnown() && !b.orig.IsNull() {
+ panic("refining non-null value as null")
+ }
+ if b.wip.null() == tristateFalse {
+ panic("refining non-null value as null")
+ }
+
+ b.wip.setNull(tristateTrue)
+
+ return b
+}
+
+// NumericRange constrains the upper and/or lower bounds of a number value,
+// or panics if this builder is not refining a number value.
+//
+// The two given values are interpreted as inclusive bounds and either one
+// may be an unknown number if only one of the two bounds is currently known.
+// If either of the given values is not a non-null number value then this
+// function will panic.
+func (b *RefinementBuilder) NumberRangeInclusive(min, max Value) *RefinementBuilder {
+ return b.NumberRangeLowerBound(min, true).NumberRangeUpperBound(max, true)
+}
+
+// NumberRangeLowerBound constraints the lower bound of a number value, or
+// panics if this builder is not refining a number value.
+func (b *RefinementBuilder) NumberRangeLowerBound(min Value, inclusive bool) *RefinementBuilder {
+ b.assertRefineable()
+
+ wip, ok := b.wip.(*refinementNumber)
+ if !ok {
+ panic(fmt.Sprintf("cannot refine numeric bounds for a %#v value", b.orig.Type()))
+ }
+
+ if !min.IsKnown() {
+ // Nothing to do if the lower bound is unknown.
+ return b
+ }
+ if min.IsNull() {
+ panic("number range lower bound must not be null")
+ }
+
+ if inclusive {
+ if gt := min.GreaterThan(b.orig); gt.IsKnown() && gt.True() {
+ panic(fmt.Sprintf("refining %#v to be >= %#v", b.orig, min))
+ }
+ } else {
+ if gt := min.GreaterThanOrEqualTo(b.orig); gt.IsKnown() && gt.True() {
+ panic(fmt.Sprintf("refining %#v to be > %#v", b.orig, min))
+ }
+ }
+
+ if wip.min != NilVal {
+ var ok Value
+ if inclusive && !wip.minInc {
+ ok = min.GreaterThan(wip.min)
+ } else {
+ ok = min.GreaterThanOrEqualTo(wip.min)
+ }
+ if ok.IsKnown() && ok.False() {
+ return b // Our existing refinement is more constrained
+ }
+ }
+
+ if min != NegativeInfinity {
+ wip.min = min
+ wip.minInc = inclusive
+ }
+
+ wip.assertConsistentBounds()
+ return b
+}
+
+// NumberRangeUpperBound constraints the upper bound of a number value, or
+// panics if this builder is not refining a number value.
+func (b *RefinementBuilder) NumberRangeUpperBound(max Value, inclusive bool) *RefinementBuilder {
+ b.assertRefineable()
+
+ wip, ok := b.wip.(*refinementNumber)
+ if !ok {
+ panic(fmt.Sprintf("cannot refine numeric bounds for a %#v value", b.orig.Type()))
+ }
+
+ if !max.IsKnown() {
+ // Nothing to do if the upper bound is unknown.
+ return b
+ }
+ if max.IsNull() {
+ panic("number range upper bound must not be null")
+ }
+
+ if inclusive {
+ if lt := max.LessThan(b.orig); lt.IsKnown() && lt.True() {
+ panic(fmt.Sprintf("refining %#v to be <= %#v", b.orig, max))
+ }
+ } else {
+ if lt := max.LessThanOrEqualTo(b.orig); lt.IsKnown() && lt.True() {
+ panic(fmt.Sprintf("refining %#v to be < %#v", b.orig, max))
+ }
+ }
+
+ if wip.max != NilVal {
+ var ok Value
+ if inclusive && !wip.maxInc {
+ ok = max.LessThan(wip.max)
+ } else {
+ ok = max.LessThanOrEqualTo(wip.max)
+ }
+ if ok.IsKnown() && ok.False() {
+ return b // Our existing refinement is more constrained
+ }
+ }
+
+ if max != PositiveInfinity {
+ wip.max = max
+ wip.maxInc = inclusive
+ }
+
+ wip.assertConsistentBounds()
+ return b
+}
+
+// CollectionLengthLowerBound constrains the lower bound of the length of a
+// collection value, or panics if this builder is not refining a collection
+// value.
+func (b *RefinementBuilder) CollectionLengthLowerBound(min int) *RefinementBuilder {
+ b.assertRefineable()
+
+ wip, ok := b.wip.(*refinementCollection)
+ if !ok {
+ panic(fmt.Sprintf("cannot refine collection length bounds for a %#v value", b.orig.Type()))
+ }
+
+ minVal := NumberIntVal(int64(min))
+ if b.orig.IsKnown() {
+ realLen := b.orig.Length()
+ if gt := minVal.GreaterThan(realLen); gt.IsKnown() && gt.True() {
+ panic(fmt.Sprintf("refining collection of length %#v with lower bound %#v", realLen, min))
+ }
+ }
+
+ if wip.minLen > min {
+ return b // Our existing refinement is more constrained
+ }
+
+ wip.minLen = min
+ wip.assertConsistentLengthBounds()
+
+ return b
+}
+
+// CollectionLengthUpperBound constrains the upper bound of the length of a
+// collection value, or panics if this builder is not refining a collection
+// value.
+//
+// The upper bound must be a known, non-null number or this function will
+// panic.
+func (b *RefinementBuilder) CollectionLengthUpperBound(max int) *RefinementBuilder {
+ b.assertRefineable()
+
+ wip, ok := b.wip.(*refinementCollection)
+ if !ok {
+ panic(fmt.Sprintf("cannot refine collection length bounds for a %#v value", b.orig.Type()))
+ }
+
+ if b.orig.IsKnown() {
+ maxVal := NumberIntVal(int64(max))
+ realLen := b.orig.Length()
+ if lt := maxVal.LessThan(realLen); lt.IsKnown() && lt.True() {
+ panic(fmt.Sprintf("refining collection of length %#v with upper bound %#v", realLen, max))
+ }
+ }
+
+ if wip.maxLen < max {
+ return b // Our existing refinement is more constrained
+ }
+
+ wip.maxLen = max
+ wip.assertConsistentLengthBounds()
+
+ return b
+}
+
+// CollectionLength is a shorthand for passing the same length to both
+// [CollectionLengthLowerBound] and [CollectionLengthUpperBound].
+//
+// A collection with a refined length with equal bounds can sometimes collapse
+// to a known value. Refining to length zero always produces a known value.
+// The behavior for other lengths varies by collection type kind.
+//
+// If the unknown value is of a set type, it's only valid to use this method
+// if the caller knows that there will be the given number of _unique_ values
+// in the set. If any values might potentially coalesce together once known,
+// use [CollectionLengthUpperBound] instead.
+func (b *RefinementBuilder) CollectionLength(length int) *RefinementBuilder {
+ return b.CollectionLengthLowerBound(length).CollectionLengthUpperBound(length)
+}
+
+// StringPrefix constrains the prefix of a string value, or panics if this
+// builder is not refining a string value.
+//
+// The given prefix will be Unicode normalized in the same way that a
+// cty.StringVal would be.
+//
+// Due to Unicode normalization and grapheme cluster rules, appending new
+// characters to a string can change the meaning of earlier characters.
+// StringPrefix may discard one or more characters from the end of the given
+// prefix to avoid that problem.
+//
+// Although cty cannot check this automatically, applications should avoid
+// relying on the discarding of the suffix for correctness. For example, if the
+// prefix ends with an emoji base character then StringPrefix will discard it
+// in case subsequent characters include emoji modifiers, but it's still
+// incorrect for the final string to use an entirely different base character.
+//
+// Applications which fully control the final result and can guarantee the
+// subsequent characters will not combine with the prefix may be able to use
+// [RefinementBuilder.StringPrefixFull] instead, after carefully reviewing
+// the constraints described in its documentation.
+func (b *RefinementBuilder) StringPrefix(prefix string) *RefinementBuilder {
+ return b.StringPrefixFull(ctystrings.SafeKnownPrefix(prefix))
+}
+
+// StringPrefixFull is a variant of StringPrefix that will never shorten the
+// given prefix to take into account the possibility of the next character
+// combining with the end of the prefix.
+//
+// Applications which fully control the subsequent characters can use this
+// as long as they guarantee that the characters added later cannot possibly
+// combine with characters at the end of the prefix to form a single grapheme
+// cluster. For example, it would be unsafe to use the full prefix "hello" if
+// there is any chance that the final string will add a combining diacritic
+// character after the "o", because that would then change the final character.
+//
+// Use [RefinementBuilder.StringPrefix] instead if an application cannot fully
+// control the final result to avoid violating this rule.
+func (b *RefinementBuilder) StringPrefixFull(prefix string) *RefinementBuilder {
+ b.assertRefineable()
+
+ wip, ok := b.wip.(*refinementString)
+ if !ok {
+ panic(fmt.Sprintf("cannot refine string prefix for a %#v value", b.orig.Type()))
+ }
+
+ // We must apply the same Unicode processing we'd normally use for a
+ // cty string so that the prefix will be comparable.
+ prefix = NormalizeString(prefix)
+
+ // If we have a known string value then the given prefix must actually
+ // match it.
+ if b.orig.IsKnown() && !b.orig.IsNull() {
+ have := b.orig.AsString()
+ matchLen := len(have)
+ if l := len(prefix); l < matchLen {
+ matchLen = l
+ }
+ have = have[:matchLen]
+ new := prefix[:matchLen]
+ if have != new {
+ panic("refined prefix is inconsistent with known value")
+ }
+ }
+
+ // If we already have a refined prefix then the overlapping parts of that
+ // and the new prefix must match.
+ {
+ matchLen := len(wip.prefix)
+ if l := len(prefix); l < matchLen {
+ matchLen = l
+ }
+
+ have := wip.prefix[:matchLen]
+ new := prefix[:matchLen]
+ if have != new {
+ panic("refined prefix is inconsistent with previous refined prefix")
+ }
+ }
+
+ // We'll only save the new prefix if it's longer than the one we already
+ // had.
+ if len(prefix) > len(wip.prefix) {
+ wip.prefix = prefix
+ }
+
+ return b
+}
+
+// NewValue completes the refinement process by constructing a new value
+// that is guaranteed to meet all of the previously-specified refinements.
+//
+// If the original value being refined was known then the result is exactly
+// that value, because otherwise the previous refinement calls would have
+// panicked reporting the refinements as invalid for the value.
+//
+// If the original value was unknown then the result is typically also unknown
+// but may have additional refinements compared to the original. If the applied
+// refinements have reduced the range to a single exact value then the result
+// might be that known value.
+func (b *RefinementBuilder) NewValue() (ret Value) {
+ defer func() {
+ // Regardless of how we return, the new value should have the same
+ // marks as our original value.
+ ret = ret.WithMarks(b.marks)
+ }()
+
+ if b.orig.IsKnown() {
+ return b.orig
+ }
+
+ // We have a few cases where the value has been refined enough that we now
+ // know exactly what the value is, or at least we can produce a more
+ // detailed approximation of it.
+ switch b.wip.null() {
+ case tristateTrue:
+ // There is only one null value of each type so this is now known.
+ return NullVal(b.orig.Type())
+ case tristateFalse:
+ // If we know it's definitely not null then we might have enough
+ // information to construct a known, non-null value.
+ if rfn, ok := b.wip.(*refinementNumber); ok {
+ // If both bounds are inclusive and equal then our value can
+ // only be the same number as the bounds.
+ if rfn.maxInc && rfn.minInc {
+ if rfn.min != NilVal && rfn.max != NilVal {
+ eq := rfn.min.Equals(rfn.max)
+ if eq.IsKnown() && eq.True() {
+ return rfn.min
+ }
+ }
+ }
+ } else if rfn, ok := b.wip.(*refinementCollection); ok {
+ // If both of the bounds are equal then we know the length is
+ // the same number as the bounds.
+ if rfn.minLen == rfn.maxLen {
+ knownLen := rfn.minLen
+ ty := b.orig.Type()
+ if knownLen == 0 {
+ // If we know the length is zero then we can construct
+ // a known value of any collection kind.
+ switch {
+ case ty.IsListType():
+ return ListValEmpty(ty.ElementType())
+ case ty.IsSetType():
+ return SetValEmpty(ty.ElementType())
+ case ty.IsMapType():
+ return MapValEmpty(ty.ElementType())
+ }
+ } else if ty.IsListType() {
+ // If we know the length of the list then we can
+ // create a known list with unknown elements instead
+ // of a wholly-unknown list.
+ elems := make([]Value, knownLen)
+ unk := UnknownVal(ty.ElementType())
+ for i := range elems {
+ elems[i] = unk
+ }
+ return ListVal(elems)
+ } else if ty.IsSetType() && knownLen == 1 {
+ // If we know we have a one-element set then we
+ // know the one element can't possibly coalesce with
+ // anything else and so we can create a known set with
+ // an unknown element.
+ return SetVal([]Value{UnknownVal(ty.ElementType())})
+ }
+ }
+ }
+ }
+
+ return Value{
+ ty: b.orig.ty,
+ v: &unknownType{refinement: b.wip},
+ }
+}
+
+// unknownValRefinment is an interface pretending to be a sum type representing
+// the different kinds of unknown value refinements we support for different
+// types of value.
+type unknownValRefinement interface {
+ unknownValRefinementSigil()
+ copy() unknownValRefinement
+ null() tristateBool
+ setNull(tristateBool)
+ rawEqual(other unknownValRefinement) bool
+ GoString() string
+}
+
+type refinementString struct {
+ refinementNullable
+ prefix string
+}
+
+func (r *refinementString) unknownValRefinementSigil() {}
+
+func (r *refinementString) copy() unknownValRefinement {
+ ret := *r
+ // Everything in refinementString is immutable, so a shallow copy is sufficient.
+ return &ret
+}
+
+func (r *refinementString) rawEqual(other unknownValRefinement) bool {
+ {
+ other, ok := other.(*refinementString)
+ if !ok {
+ return false
+ }
+ return (r.refinementNullable.rawEqual(&other.refinementNullable) &&
+ r.prefix == other.prefix)
+ }
+}
+
+func (r *refinementString) GoString() string {
+ var b strings.Builder
+ b.WriteString(r.refinementNullable.GoString())
+ if r.prefix != "" {
+ fmt.Fprintf(&b, ".StringPrefixFull(%q)", r.prefix)
+ }
+ return b.String()
+}
+
+type refinementNumber struct {
+ refinementNullable
+ min, max Value
+ minInc, maxInc bool
+}
+
+func (r *refinementNumber) unknownValRefinementSigil() {}
+
+func (r *refinementNumber) copy() unknownValRefinement {
+ ret := *r
+ // Everything in refinementNumber is immutable, so a shallow copy is sufficient.
+ return &ret
+}
+
+func (r *refinementNumber) rawEqual(other unknownValRefinement) bool {
+ {
+ other, ok := other.(*refinementNumber)
+ if !ok {
+ return false
+ }
+ return (r.refinementNullable.rawEqual(&other.refinementNullable) &&
+ r.min.RawEquals(other.min) &&
+ r.max.RawEquals(other.max) &&
+ r.minInc == other.minInc &&
+ r.maxInc == other.maxInc)
+ }
+}
+
+func (r *refinementNumber) GoString() string {
+ var b strings.Builder
+ b.WriteString(r.refinementNullable.GoString())
+ if r.min != NilVal && r.min != NegativeInfinity {
+ fmt.Fprintf(&b, ".NumberLowerBound(%#v, %t)", r.min, r.minInc)
+ }
+ if r.max != NilVal && r.max != PositiveInfinity {
+ fmt.Fprintf(&b, ".NumberUpperBound(%#v, %t)", r.max, r.maxInc)
+ }
+ return b.String()
+}
+
+func (r *refinementNumber) assertConsistentBounds() {
+ if r.min == NilVal || r.max == NilVal {
+ return // If only one bound is constrained then there's nothing to be inconsistent with
+ }
+ var ok Value
+ if r.minInc != r.maxInc {
+ ok = r.min.LessThan(r.max)
+ } else {
+ ok = r.min.LessThanOrEqualTo(r.max)
+ }
+ if ok.IsKnown() && ok.False() {
+ panic(fmt.Sprintf("number lower bound %#v is greater than upper bound %#v", r.min, r.max))
+ }
+}
+
+type refinementCollection struct {
+ refinementNullable
+ minLen, maxLen int
+}
+
+func (r *refinementCollection) unknownValRefinementSigil() {}
+
+func (r *refinementCollection) copy() unknownValRefinement {
+ ret := *r
+ // Everything in refinementCollection is immutable, so a shallow copy is sufficient.
+ return &ret
+}
+
+func (r *refinementCollection) rawEqual(other unknownValRefinement) bool {
+ {
+ other, ok := other.(*refinementCollection)
+ if !ok {
+ return false
+ }
+ return (r.refinementNullable.rawEqual(&other.refinementNullable) &&
+ r.minLen == other.minLen &&
+ r.maxLen == other.maxLen)
+ }
+}
+
+func (r *refinementCollection) GoString() string {
+ var b strings.Builder
+ b.WriteString(r.refinementNullable.GoString())
+ if r.minLen != 0 {
+ fmt.Fprintf(&b, ".CollectionLengthLowerBound(%d)", r.minLen)
+ }
+ if r.maxLen != math.MaxInt {
+ fmt.Fprintf(&b, ".CollectionLengthUpperBound(%d)", r.maxLen)
+ }
+ return b.String()
+}
+
+func (r *refinementCollection) assertConsistentLengthBounds() {
+ if r.maxLen < r.minLen {
+ panic(fmt.Sprintf("collection length upper bound %d is less than lower bound %d", r.maxLen, r.minLen))
+ }
+}
+
+type refinementNullable struct {
+ isNull tristateBool
+}
+
+func (r *refinementNullable) unknownValRefinementSigil() {}
+
+func (r *refinementNullable) copy() unknownValRefinement {
+ ret := *r
+ // Everything in refinementJustNull is immutable, so a shallow copy is sufficient.
+ return &ret
+}
+
+func (r *refinementNullable) null() tristateBool {
+ return r.isNull
+}
+
+func (r *refinementNullable) setNull(v tristateBool) {
+ r.isNull = v
+}
+
+func (r *refinementNullable) rawEqual(other unknownValRefinement) bool {
+ {
+ other, ok := other.(*refinementNullable)
+ if !ok {
+ return false
+ }
+ return r.isNull == other.isNull
+ }
+}
+
+func (r *refinementNullable) GoString() string {
+ switch r.isNull {
+ case tristateFalse:
+ return ".NotNull()"
+ case tristateTrue:
+ return ".Null()"
+ default:
+ return ""
+ }
+}
+
+type tristateBool rune
+
+const tristateTrue tristateBool = 'T'
+const tristateFalse tristateBool = 'F'
+const tristateUnknown tristateBool = 0
diff --git a/cty/value.go b/cty/value.go
index f6a25dd..e5b29b6 100644
--- a/cty/value.go
+++ b/cty/value.go
@@ -48,7 +48,8 @@ func (val Value) IsKnown() bool {
if val.IsMarked() {
return val.unmarkForce().IsKnown()
}
- return val.v != unknown
+ _, unknown := val.v.(*unknownType)
+ return !unknown
}
// IsNull returns true if the value is null. Values of any type can be
diff --git a/cty/value_init.go b/cty/value_init.go
index 6dcae27..a1743a0 100644
--- a/cty/value_init.go
+++ b/cty/value_init.go
@@ -5,8 +5,7 @@ import (
"math/big"
"reflect"
- "golang.org/x/text/unicode/norm"
-
+ "github.com/zclconf/go-cty/cty/ctystrings"
"github.com/zclconf/go-cty/cty/set"
)
@@ -107,7 +106,7 @@ func StringVal(v string) Value {
// A return value from this function can be meaningfully compared byte-for-byte
// with a Value.AsString result.
func NormalizeString(s string) string {
- return norm.NFC.String(s)
+ return ctystrings.Normalize(s)
}
// ObjectVal returns a Value of an object type whose structure is defined
diff --git a/cty/value_ops.go b/cty/value_ops.go
index 88b3637..c4584bd 100644
--- a/cty/value_ops.go
+++ b/cty/value_ops.go
@@ -33,7 +33,17 @@ func (val Value) GoString() string {
return "cty.DynamicVal"
}
if !val.IsKnown() {
- return fmt.Sprintf("cty.UnknownVal(%#v)", val.ty)
+ rfn := val.v.(*unknownType).refinement
+ var suffix string
+ if rfn != nil {
+ calls := rfn.GoString()
+ if calls == ".NotNull()" {
+ suffix = ".RefineNotNull()"
+ } else {
+ suffix = ".Refine()" + rfn.GoString() + ".NewValue()"
+ }
+ }
+ return fmt.Sprintf("cty.UnknownVal(%#v)%s", val.ty, suffix)
}
// By the time we reach here we've dealt with all of the exceptions around
@@ -125,13 +135,38 @@ func (val Value) Equals(other Value) Value {
return val.Equals(other).WithMarks(valMarks, otherMarks)
}
- // Start by handling Unknown values before considering types.
- // This needs to be done since Null values are always equal regardless of
- // type.
+ // Some easy cases with comparisons to null.
+ switch {
+ case val.IsNull() && definitelyNotNull(other):
+ return False
+ case other.IsNull() && definitelyNotNull(val):
+ return False
+ }
+ // If we have one known value and one unknown value then we may be
+ // able to quickly disqualify equality based on the range of the unknown
+ // value.
+ if val.IsKnown() && !other.IsKnown() {
+ otherRng := other.Range()
+ if ok := otherRng.Includes(val); ok.IsKnown() && ok.False() {
+ return False
+ }
+ } else if other.IsKnown() && !val.IsKnown() {
+ valRng := val.Range()
+ if ok := valRng.Includes(other); ok.IsKnown() && ok.False() {
+ return False
+ }
+ }
+
+ // We need to deal with unknown values before anything else with nulls
+ // because any unknown value that hasn't yet been refined as non-null
+ // could become null, and nulls of any types are equal to one another.
+ unknownResult := func() Value {
+ return UnknownVal(Bool).Refine().NotNull().NewValue()
+ }
switch {
case !val.IsKnown() && !other.IsKnown():
// both unknown
- return UnknownVal(Bool)
+ return unknownResult()
case val.IsKnown() && !other.IsKnown():
switch {
case val.IsNull(), other.ty.HasDynamicTypes():
@@ -139,13 +174,13 @@ func (val Value) Equals(other Value) Value {
// nulls of any type are equal.
// An unknown with a dynamic type compares as unknown, which we need
// to check before the type comparison below.
- return UnknownVal(Bool)
+ return unknownResult()
case !val.ty.Equals(other.ty):
// There is no null comparison or dynamic types, so unequal types
// will never be equal.
return False
default:
- return UnknownVal(Bool)
+ return unknownResult()
}
case other.IsKnown() && !val.IsKnown():
switch {
@@ -154,13 +189,13 @@ func (val Value) Equals(other Value) Value {
// nulls of any type are equal.
// An unknown with a dynamic type compares as unknown, which we need
// to check before the type comparison below.
- return UnknownVal(Bool)
+ return unknownResult()
case !other.ty.Equals(val.ty):
// There's no null comparison or dynamic types, so unequal types
// will never be equal.
return False
default:
- return UnknownVal(Bool)
+ return unknownResult()
}
}
@@ -182,7 +217,7 @@ func (val Value) Equals(other Value) Value {
return BoolVal(false)
}
- return UnknownVal(Bool)
+ return unknownResult()
}
if !val.ty.Equals(other.ty) {
@@ -216,7 +251,7 @@ func (val Value) Equals(other Value) Value {
}
eq := lhs.Equals(rhs)
if !eq.IsKnown() {
- return UnknownVal(Bool)
+ return unknownResult()
}
if eq.False() {
result = false
@@ -237,7 +272,7 @@ func (val Value) Equals(other Value) Value {
}
eq := lhs.Equals(rhs)
if !eq.IsKnown() {
- return UnknownVal(Bool)
+ return unknownResult()
}
if eq.False() {
result = false
@@ -259,7 +294,7 @@ func (val Value) Equals(other Value) Value {
}
eq := lhs.Equals(rhs)
if !eq.IsKnown() {
- return UnknownVal(Bool)
+ return unknownResult()
}
if eq.False() {
result = false
@@ -276,8 +311,8 @@ func (val Value) Equals(other Value) Value {
// in one are also in the other.
for it := s1.Iterator(); it.Next(); {
rv := it.Value()
- if rv == unknown { // "unknown" is the internal representation of unknown-ness
- return UnknownVal(Bool)
+ if _, unknown := rv.(*unknownType); unknown { // "*unknownType" is the internal representation of unknown-ness
+ return unknownResult()
}
if !s2.Has(rv) {
equal = false
@@ -285,8 +320,8 @@ func (val Value) Equals(other Value) Value {
}
for it := s2.Iterator(); it.Next(); {
rv := it.Value()
- if rv == unknown { // "unknown" is the internal representation of unknown-ness
- return UnknownVal(Bool)
+ if _, unknown := rv.(*unknownType); unknown { // "*unknownType" is the internal representation of unknown-ness
+ return unknownResult()
}
if !s1.Has(rv) {
equal = false
@@ -313,7 +348,7 @@ func (val Value) Equals(other Value) Value {
}
eq := lhs.Equals(rhs)
if !eq.IsKnown() {
- return UnknownVal(Bool)
+ return unknownResult()
}
if eq.False() {
result = false
@@ -393,7 +428,17 @@ func (val Value) RawEquals(other Value) bool {
other = other.unmarkForce()
if (!val.IsKnown()) && (!other.IsKnown()) {
- return true
+ // If either unknown value has refinements then they must match.
+ valRfn := val.v.(*unknownType).refinement
+ otherRfn := other.v.(*unknownType).refinement
+ switch {
+ case (valRfn == nil) != (otherRfn == nil):
+ return false
+ case valRfn != nil:
+ return valRfn.rawEqual(otherRfn)
+ default:
+ return true
+ }
}
if (val.IsKnown() && !other.IsKnown()) || (other.IsKnown() && !val.IsKnown()) {
return false
@@ -548,7 +593,8 @@ func (val Value) Add(other Value) Value {
if shortCircuit := mustTypeCheck(Number, Number, val, other); shortCircuit != nil {
shortCircuit = forceShortCircuitType(shortCircuit, Number)
- return *shortCircuit
+ ret := shortCircuit.RefineWith(numericRangeArithmetic(Value.Add, val.Range(), other.Range()))
+ return ret.RefineNotNull()
}
ret := new(big.Float)
@@ -567,7 +613,8 @@ func (val Value) Subtract(other Value) Value {
if shortCircuit := mustTypeCheck(Number, Number, val, other); shortCircuit != nil {
shortCircuit = forceShortCircuitType(shortCircuit, Number)
- return *shortCircuit
+ ret := shortCircuit.RefineWith(numericRangeArithmetic(Value.Subtract, val.Range(), other.Range()))
+ return ret.RefineNotNull()
}
return val.Add(other.Negate())
@@ -583,7 +630,7 @@ func (val Value) Negate() Value {
if shortCircuit := mustTypeCheck(Number, Number, val); shortCircuit != nil {
shortCircuit = forceShortCircuitType(shortCircuit, Number)
- return *shortCircuit
+ return (*shortCircuit).RefineNotNull()
}
ret := new(big.Float).Neg(val.v.(*big.Float))
@@ -600,8 +647,14 @@ func (val Value) Multiply(other Value) Value {
}
if shortCircuit := mustTypeCheck(Number, Number, val, other); shortCircuit != nil {
+ // If either value is exactly zero then the result must either be
+ // zero or an error.
+ if val == Zero || other == Zero {
+ return Zero
+ }
shortCircuit = forceShortCircuitType(shortCircuit, Number)
- return *shortCircuit
+ ret := shortCircuit.RefineWith(numericRangeArithmetic(Value.Multiply, val.Range(), other.Range()))
+ return ret.RefineNotNull()
}
// find the larger precision of the arguments
@@ -646,7 +699,10 @@ func (val Value) Divide(other Value) Value {
if shortCircuit := mustTypeCheck(Number, Number, val, other); shortCircuit != nil {
shortCircuit = forceShortCircuitType(shortCircuit, Number)
- return *shortCircuit
+ // TODO: We could potentially refine the range of the result here, but
+ // we don't right now because our division operation is not monotone
+ // if the denominator could potentially be zero.
+ return (*shortCircuit).RefineNotNull()
}
ret := new(big.Float)
@@ -678,7 +734,7 @@ func (val Value) Modulo(other Value) Value {
if shortCircuit := mustTypeCheck(Number, Number, val, other); shortCircuit != nil {
shortCircuit = forceShortCircuitType(shortCircuit, Number)
- return *shortCircuit
+ return (*shortCircuit).RefineNotNull()
}
// We cheat a bit here with infinities, just abusing the Multiply operation
@@ -716,7 +772,7 @@ func (val Value) Absolute() Value {
if shortCircuit := mustTypeCheck(Number, Number, val); shortCircuit != nil {
shortCircuit = forceShortCircuitType(shortCircuit, Number)
- return *shortCircuit
+ return (*shortCircuit).Refine().NotNull().NumberRangeInclusive(Zero, UnknownVal(Number)).NewValue()
}
ret := (&big.Float{}).Abs(val.v.(*big.Float))
@@ -889,23 +945,23 @@ func (val Value) HasIndex(key Value) Value {
}
if val.ty == DynamicPseudoType {
- return UnknownVal(Bool)
+ return UnknownVal(Bool).RefineNotNull()
}
switch {
case val.Type().IsListType():
if key.Type() == DynamicPseudoType {
- return UnknownVal(Bool)
+ return UnknownVal(Bool).RefineNotNull()
}
if key.Type() != Number {
return False
}
if !key.IsKnown() {
- return UnknownVal(Bool)
+ return UnknownVal(Bool).RefineNotNull()
}
if !val.IsKnown() {
- return UnknownVal(Bool)
+ return UnknownVal(Bool).RefineNotNull()
}
index, accuracy := key.v.(*big.Float).Int64()
@@ -916,17 +972,17 @@ func (val Value) HasIndex(key Value) Value {
return BoolVal(int(index) < len(val.v.([]interface{})) && index >= 0)
case val.Type().IsMapType():
if key.Type() == DynamicPseudoType {
- return UnknownVal(Bool)
+ return UnknownVal(Bool).RefineNotNull()
}
if key.Type() != String {
return False
}
if !key.IsKnown() {
- return UnknownVal(Bool)
+ return UnknownVal(Bool).RefineNotNull()
}
if !val.IsKnown() {
- return UnknownVal(Bool)
+ return UnknownVal(Bool).RefineNotNull()
}
keyStr := key.v.(string)
@@ -935,14 +991,14 @@ func (val Value) HasIndex(key Value) Value {
return BoolVal(exists)
case val.Type().IsTupleType():
if key.Type() == DynamicPseudoType {
- return UnknownVal(Bool)
+ return UnknownVal(Bool).RefineNotNull()
}
if key.Type() != Number {
return False
}
if !key.IsKnown() {
- return UnknownVal(Bool)
+ return UnknownVal(Bool).RefineNotNull()
}
index, accuracy := key.v.(*big.Float).Int64()
@@ -977,10 +1033,10 @@ func (val Value) HasElement(elem Value) Value {
panic("not a set type")
}
if !val.IsKnown() || !elem.IsKnown() {
- return UnknownVal(Bool)
+ return UnknownVal(Bool).RefineNotNull()
}
if val.IsNull() {
- panic("can't call HasElement on a nil value")
+ panic("can't call HasElement on a null value")
}
if !ty.ElementType().Equals(elem.Type()) {
return False
@@ -1012,7 +1068,10 @@ func (val Value) Length() Value {
}
if !val.IsKnown() {
- return UnknownVal(Number)
+ // If the whole collection isn't known then the length isn't known
+ // either, but we can still put some bounds on the range of the result.
+ rng := val.Range()
+ return UnknownVal(Number).RefineWith(valueRefineLengthResult(rng))
}
if val.Type().IsSetType() {
// The Length rules are a little different for sets because if any
@@ -1030,13 +1089,26 @@ func (val Value) Length() Value {
// unknown value cannot represent more than one known value.
return NumberIntVal(storeLength)
}
- // Otherwise, we cannot predict the length.
- return UnknownVal(Number)
+ // Otherwise, we cannot predict the length exactly but we can at
+ // least constrain both bounds of its range, because value coalescing
+ // can only ever reduce the number of elements in the set.
+ return UnknownVal(Number).Refine().NotNull().NumberRangeInclusive(NumberIntVal(1), NumberIntVal(storeLength)).NewValue()
}
return NumberIntVal(int64(val.LengthInt()))
}
+func valueRefineLengthResult(collRng ValueRange) func(*RefinementBuilder) *RefinementBuilder {
+ return func(b *RefinementBuilder) *RefinementBuilder {
+ return b.
+ NotNull().
+ NumberRangeInclusive(
+ NumberIntVal(int64(collRng.LengthLowerBound())),
+ NumberIntVal(int64(collRng.LengthUpperBound())),
+ )
+ }
+}
+
// LengthInt is like Length except it returns an int. It has the same behavior
// as Length except that it will panic if the receiver is unknown.
//
@@ -1167,7 +1239,7 @@ func (val Value) Not() Value {
if shortCircuit := mustTypeCheck(Bool, Bool, val); shortCircuit != nil {
shortCircuit = forceShortCircuitType(shortCircuit, Bool)
- return *shortCircuit
+ return (*shortCircuit).RefineNotNull()
}
return BoolVal(!val.v.(bool))
@@ -1183,8 +1255,14 @@ func (val Value) And(other Value) Value {
}
if shortCircuit := mustTypeCheck(Bool, Bool, val, other); shortCircuit != nil {
+ // If either value is known to be exactly False then it doesn't
+ // matter what the other value is, because the final result must
+ // either be False or an error.
+ if val == False || other == False {
+ return False
+ }
shortCircuit = forceShortCircuitType(shortCircuit, Bool)
- return *shortCircuit
+ return (*shortCircuit).RefineNotNull()
}
return BoolVal(val.v.(bool) && other.v.(bool))
@@ -1200,8 +1278,14 @@ func (val Value) Or(other Value) Value {
}
if shortCircuit := mustTypeCheck(Bool, Bool, val, other); shortCircuit != nil {
+ // If either value is known to be exactly True then it doesn't
+ // matter what the other value is, because the final result must
+ // either be True or an error.
+ if val == True || other == True {
+ return True
+ }
shortCircuit = forceShortCircuitType(shortCircuit, Bool)
- return *shortCircuit
+ return (*shortCircuit).RefineNotNull()
}
return BoolVal(val.v.(bool) || other.v.(bool))
@@ -1217,8 +1301,30 @@ func (val Value) LessThan(other Value) Value {
}
if shortCircuit := mustTypeCheck(Number, Bool, val, other); shortCircuit != nil {
+ // We might be able to return a known answer even with unknown inputs.
+ // FIXME: This is more conservative than it needs to be, because it
+ // treats all bounds as exclusive bounds.
+ valRng := val.Range()
+ otherRng := other.Range()
+ if valRng.TypeConstraint() == Number && other.Range().TypeConstraint() == Number {
+ valMax, _ := valRng.NumberUpperBound()
+ otherMin, _ := otherRng.NumberLowerBound()
+ if valMax.IsKnown() && otherMin.IsKnown() {
+ if r := valMax.LessThan(otherMin); r.True() {
+ return True
+ }
+ }
+ valMin, _ := valRng.NumberLowerBound()
+ otherMax, _ := otherRng.NumberUpperBound()
+ if valMin.IsKnown() && otherMax.IsKnown() {
+ if r := valMin.GreaterThan(otherMax); r.True() {
+ return False
+ }
+ }
+ }
+
shortCircuit = forceShortCircuitType(shortCircuit, Bool)
- return *shortCircuit
+ return (*shortCircuit).RefineNotNull()
}
return BoolVal(val.v.(*big.Float).Cmp(other.v.(*big.Float)) < 0)
@@ -1234,8 +1340,30 @@ func (val Value) GreaterThan(other Value) Value {
}
if shortCircuit := mustTypeCheck(Number, Bool, val, other); shortCircuit != nil {
+ // We might be able to return a known answer even with unknown inputs.
+ // FIXME: This is more conservative than it needs to be, because it
+ // treats all bounds as exclusive bounds.
+ valRng := val.Range()
+ otherRng := other.Range()
+ if valRng.TypeConstraint() == Number && other.Range().TypeConstraint() == Number {
+ valMin, _ := valRng.NumberLowerBound()
+ otherMax, _ := otherRng.NumberUpperBound()
+ if valMin.IsKnown() && otherMax.IsKnown() {
+ if r := valMin.GreaterThan(otherMax); r.True() {
+ return True
+ }
+ }
+ valMax, _ := valRng.NumberUpperBound()
+ otherMin, _ := otherRng.NumberLowerBound()
+ if valMax.IsKnown() && otherMin.IsKnown() {
+ if r := valMax.LessThan(otherMin); r.True() {
+ return False
+ }
+ }
+ }
+
shortCircuit = forceShortCircuitType(shortCircuit, Bool)
- return *shortCircuit
+ return (*shortCircuit).RefineNotNull()
}
return BoolVal(val.v.(*big.Float).Cmp(other.v.(*big.Float)) > 0)
diff --git a/cty/value_ops_test.go b/cty/value_ops_test.go
index 1d41183..d3ad902 100644
--- a/cty/value_ops_test.go
+++ b/cty/value_ops_test.go
@@ -11,6 +11,8 @@ func TestValueEquals(t *testing.T) {
capsuleB := CapsuleVal(capsuleTestType1, &capsuleTestType1Native{"capsuleB"})
capsuleC := CapsuleVal(capsuleTestType2, &capsuleTestType2Native{"capsuleC"})
+ unknownResult := UnknownVal(Bool).RefineNotNull()
+
tests := []struct {
LHS Value
RHS Value
@@ -250,47 +252,47 @@ func TestValueEquals(t *testing.T) {
{
TupleVal([]Value{UnknownVal(Number)}),
TupleVal([]Value{NumberIntVal(1)}),
- UnknownVal(Bool),
+ unknownResult,
},
{
TupleVal([]Value{UnknownVal(Number)}),
TupleVal([]Value{UnknownVal(Number)}),
- UnknownVal(Bool),
+ unknownResult,
},
{
TupleVal([]Value{NumberIntVal(1)}),
TupleVal([]Value{UnknownVal(Number)}),
- UnknownVal(Bool),
+ unknownResult,
},
{
TupleVal([]Value{NumberIntVal(1)}),
TupleVal([]Value{DynamicVal}),
- UnknownVal(Bool),
+ unknownResult,
},
{
TupleVal([]Value{DynamicVal}),
TupleVal([]Value{NumberIntVal(1)}),
- UnknownVal(Bool),
+ unknownResult,
},
{
TupleVal([]Value{NumberIntVal(1)}),
UnknownVal(Tuple([]Type{Number})),
- UnknownVal(Bool),
+ unknownResult,
},
{
UnknownVal(Tuple([]Type{Number})),
TupleVal([]Value{NumberIntVal(1)}),
- UnknownVal(Bool),
+ unknownResult,
},
{
DynamicVal,
TupleVal([]Value{NumberIntVal(1)}),
- UnknownVal(Bool),
+ unknownResult,
},
{
TupleVal([]Value{NumberIntVal(1)}),
DynamicVal,
- UnknownVal(Bool),
+ unknownResult,
},
// Lists
@@ -532,7 +534,7 @@ func TestValueEquals(t *testing.T) {
SetVal([]Value{
UnknownVal(Number),
}),
- UnknownVal(Bool),
+ unknownResult,
},
{
SetVal([]Value{
@@ -542,7 +544,7 @@ func TestValueEquals(t *testing.T) {
NumberIntVal(1),
UnknownVal(Number),
}),
- UnknownVal(Bool),
+ unknownResult,
},
{
SetVal([]Value{
@@ -552,7 +554,7 @@ func TestValueEquals(t *testing.T) {
SetVal([]Value{
NumberIntVal(1),
}),
- UnknownVal(Bool),
+ unknownResult,
},
// Capsules
@@ -574,7 +576,7 @@ func TestValueEquals(t *testing.T) {
{
capsuleA,
UnknownVal(capsuleTestType1), // same type
- UnknownVal(Bool),
+ unknownResult,
},
{
capsuleA,
@@ -586,22 +588,32 @@ func TestValueEquals(t *testing.T) {
{
NumberIntVal(2),
UnknownVal(Number),
- UnknownVal(Bool),
+ unknownResult,
},
{
NumberIntVal(1),
DynamicVal,
- UnknownVal(Bool),
+ unknownResult,
+ },
+ {
+ NumberIntVal(2),
+ UnknownVal(Number).Refine().NumberRangeLowerBound(Zero, true).NewValue(),
+ unknownResult,
+ },
+ {
+ NumberIntVal(2),
+ UnknownVal(Number).Refine().NumberRangeLowerBound(NumberIntVal(4), true).NewValue(),
+ False, // deduction from refinement
},
{
DynamicVal,
BoolVal(true),
- UnknownVal(Bool),
+ unknownResult,
},
{
DynamicVal,
DynamicVal,
- UnknownVal(Bool),
+ unknownResult,
},
{
ListVal([]Value{
@@ -612,7 +624,7 @@ func TestValueEquals(t *testing.T) {
StringVal("hi"),
DynamicVal,
}),
- UnknownVal(Bool),
+ unknownResult,
},
{
ListVal([]Value{
@@ -623,7 +635,12 @@ func TestValueEquals(t *testing.T) {
StringVal("hi"),
UnknownVal(String),
}),
- UnknownVal(Bool),
+ unknownResult,
+ },
+ {
+ UnknownVal(List(String)).Refine().CollectionLengthLowerBound(1).NewValue(),
+ ListValEmpty(String),
+ False, // deduction from refinement
},
{
MapVal(map[string]Value{
@@ -634,7 +651,7 @@ func TestValueEquals(t *testing.T) {
"static": StringVal("hi"),
"dynamic": DynamicVal,
}),
- UnknownVal(Bool),
+ unknownResult,
},
{
MapVal(map[string]Value{
@@ -645,7 +662,7 @@ func TestValueEquals(t *testing.T) {
"static": StringVal("hi"),
"dynamic": UnknownVal(String),
}),
- UnknownVal(Bool),
+ unknownResult,
},
{
NullVal(String),
@@ -660,7 +677,7 @@ func TestValueEquals(t *testing.T) {
{
UnknownVal(String),
UnknownVal(Number),
- UnknownVal(Bool),
+ unknownResult,
},
{
StringVal(""),
@@ -675,7 +692,7 @@ func TestValueEquals(t *testing.T) {
{
StringVal(""),
UnknownVal(String),
- UnknownVal(Bool),
+ unknownResult,
},
{
NullVal(DynamicPseudoType),
@@ -685,17 +702,17 @@ func TestValueEquals(t *testing.T) {
{
NullVal(String),
UnknownVal(Number),
- UnknownVal(Bool), // because second operand might eventually be null
+ unknownResult, // because second operand might eventually be null
},
{
UnknownVal(String),
NullVal(Number),
- UnknownVal(Bool), // because first operand might eventually be null
+ unknownResult, // because first operand might eventually be null
},
{
UnknownVal(String),
UnknownVal(Number),
- UnknownVal(Bool), // because both operands might eventually be null
+ unknownResult, // because both operands might eventually be null
},
{
StringVal("hello"),
@@ -759,7 +776,7 @@ func TestValueEquals(t *testing.T) {
ObjectVal(map[string]Value{
"a": DynamicVal,
}),
- UnknownVal(Bool),
+ unknownResult,
},
{
ObjectVal(map[string]Value{
@@ -779,7 +796,47 @@ func TestValueEquals(t *testing.T) {
ObjectVal(map[string]Value{
"a": UnknownVal(List(List(DynamicPseudoType))),
}),
- UnknownVal(Bool),
+ unknownResult,
+ },
+ {
+ NullVal(String),
+ UnknownVal(String).Refine().NotNull().NewValue(),
+ False,
+ },
+ {
+ UnknownVal(String).Refine().NotNull().NewValue(),
+ NullVal(String),
+ False,
+ },
+ {
+ UnknownVal(String).Refine().Null().NewValue(),
+ NullVal(String),
+ True, // NOTE: The refinement should collapse to NullVal(String)
+ },
+ {
+ NullVal(String),
+ UnknownVal(String).Refine().Null().NewValue(),
+ True, // NOTE: The refinement should collapse to NullVal(String)
+ },
+ {
+ UnknownVal(String).Refine().StringPrefix("foo-").NewValue(),
+ StringVal("notfoo-bar"),
+ False, // Deduction from refinement
+ },
+ {
+ StringVal(""),
+ UnknownVal(String).Refine().StringPrefix("foo-").NewValue(),
+ False, // Deduction from refinement
+ },
+ {
+ StringVal("").Mark("a"),
+ UnknownVal(String).Mark("b").Refine().StringPrefix("foo-").NewValue(),
+ False.Mark("a").Mark("b"), // Deduction from refinement
+ },
+ {
+ UnknownVal(String).Refine().StringPrefix("foo-").NewValue(),
+ StringVal("foo-bar"),
+ unknownResult,
},
// Marks
@@ -814,7 +871,7 @@ func TestValueEquals(t *testing.T) {
t.Run(fmt.Sprintf("%#v.Equals(%#v)", test.LHS, test.RHS), func(t *testing.T) {
got := test.LHS.Equals(test.RHS)
if !got.RawEquals(test.Expected) {
- t.Fatalf("wrong Equals result\ngot: %#v\nwant: %#v", got, test.Expected)
+ t.Fatalf("wrong Equals result\nLHS: %#v\nRHS: %#v\ngot: %#v\nwant: %#v", test.LHS, test.RHS, got, test.Expected)
}
})
}
@@ -1595,6 +1652,67 @@ func TestValueRawEquals(t *testing.T) {
}),
false,
},
+ {
+ UnknownVal(Bool).Refine().NotNull().NewValue(),
+ UnknownVal(Bool),
+ false,
+ },
+ {
+ UnknownVal(Bool),
+ UnknownVal(Bool).Refine().NotNull().NewValue(),
+ false,
+ },
+ {
+ UnknownVal(Number).Refine().NumberRangeInclusive(Zero, NumberIntVal(1)).NewValue(),
+ UnknownVal(Number).Refine().NumberRangeInclusive(Zero, NumberIntVal(2)).NewValue(),
+ false,
+ },
+ {
+ UnknownVal(Number).Refine().NumberRangeInclusive(Zero, NumberIntVal(1)).NewValue(),
+ UnknownVal(Number).Refine().NumberRangeInclusive(Zero, NumberIntVal(1)).NewValue(),
+ true,
+ },
+ {
+ UnknownVal(String),
+ UnknownVal(String).Refine().StringPrefix("foo").NewValue(),
+ false,
+ },
+ {
+ UnknownVal(String).Refine().StringPrefix("foo").NewValue(),
+ UnknownVal(String).Refine().StringPrefix("foo").NewValue(),
+ true,
+ },
+ {
+ UnknownVal(String).Refine().NotNull().StringPrefix("foo").NewValue(),
+ UnknownVal(String).Refine().StringPrefix("foo").NewValue(),
+ false,
+ },
+ {
+ UnknownVal(String).Refine().StringPrefix("foo").NewValue(),
+ UnknownVal(String).Refine().StringPrefix("bar").NewValue(),
+ false,
+ },
+ {
+ UnknownVal(String).Refine().Null().NewValue(),
+ NullVal(String),
+ true, // The refinement expression collapses into a simple null
+ },
+ {
+ UnknownVal(Number).Refine().NumberRangeInclusive(Zero, Zero).NewValue(),
+ Zero,
+ false, // Refinement can't collapse to zero because it might be null
+ },
+ {
+ UnknownVal(Number).Refine().NotNull().NumberRangeInclusive(Zero, Zero).NewValue(),
+ Zero,
+ true, // Refinement collapses to zero because it's not null and the two bounds are equal
+ },
+ {
+ UnknownVal(List(String)).Refine().NotNull().CollectionLengthUpperBound(0).NewValue(),
+ ListValEmpty(String),
+ true, // Colection length lower bound is always at least zero so this refinement collapses to an empty list
+ },
+
// Marks
{
StringVal("a").Mark(1),
@@ -1617,7 +1735,7 @@ func TestValueRawEquals(t *testing.T) {
t.Run(fmt.Sprintf("%#v.RawEquals(%#v)", test.LHS, test.RHS), func(t *testing.T) {
got := test.LHS.RawEquals(test.RHS)
if !got == test.Expected {
- t.Fatalf("wrong Equals result\ngot: %#v\nwant: %#v", got, test.Expected)
+ t.Fatalf("wrong RawEquals result\ngot: %#v\nwant: %#v", got, test.Expected)
}
})
}
@@ -1647,22 +1765,82 @@ func TestValueAdd(t *testing.T) {
{
NumberIntVal(1),
UnknownVal(Number),
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
+ },
+ {
+ NumberIntVal(1),
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeLowerBound(NumberIntVal(3), true).
+ NewValue(),
+ },
+ {
+ Zero,
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeLowerBound(NumberIntVal(2), true).
+ NewValue(),
+ },
+ {
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeLowerBound(NumberIntVal(4), true).
+ NewValue(),
+ },
+ {
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(1), true).
+ NumberRangeUpperBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeLowerBound(NumberIntVal(3), true).
+ NewValue(),
+ },
+ {
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(1), true).
+ NumberRangeUpperBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), false).
+ NumberRangeUpperBound(NumberIntVal(3), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeLowerBound(NumberIntVal(3), true).
+ NumberRangeUpperBound(NumberIntVal(5), true).
+ NewValue(),
},
{
UnknownVal(Number),
UnknownVal(Number),
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
NumberIntVal(1),
DynamicVal,
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
DynamicVal,
DynamicVal,
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
Zero.Mark(1),
@@ -1685,7 +1863,7 @@ func TestValueAdd(t *testing.T) {
t.Run(fmt.Sprintf("%#v.Add(%#v)", test.LHS, test.RHS), func(t *testing.T) {
got := test.LHS.Add(test.RHS)
if !got.RawEquals(test.Expected) {
- t.Fatalf("Add returned %#v; want %#v", got, test.Expected)
+ t.Fatalf("Wrong result\ngot: %#v\nwant: %#v", got, test.Expected)
}
})
}
@@ -1715,22 +1893,79 @@ func TestValueSubtract(t *testing.T) {
{
NumberIntVal(1),
UnknownVal(Number),
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
UnknownVal(Number),
UnknownVal(Number),
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
+ },
+ {
+ NumberIntVal(1),
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), true).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeUpperBound(NumberIntVal(-1), true).
+ NewValue(),
+ },
+ {
+ Zero,
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), true).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeUpperBound(NumberIntVal(-2), true).
+ NewValue(),
+ },
+ {
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), true).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), true).
+ NewValue(),
+ UnknownVal(Number).RefineNotNull(), // We don't currently refine this case
+ },
+ {
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(1), true).
+ NumberRangeUpperBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), true).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeUpperBound(NumberIntVal(0), true).
+ NewValue(),
+ },
+ {
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(1), true).
+ NumberRangeUpperBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), false).
+ NumberRangeUpperBound(NumberIntVal(3), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeLowerBound(NumberIntVal(-2), true).
+ NumberRangeUpperBound(NumberIntVal(0), true).
+ NewValue(),
},
{
NumberIntVal(1),
DynamicVal,
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
DynamicVal,
DynamicVal,
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
Zero.Mark(1),
@@ -1753,7 +1988,7 @@ func TestValueSubtract(t *testing.T) {
t.Run(fmt.Sprintf("%#v.Subtract(%#v)", test.LHS, test.RHS), func(t *testing.T) {
got := test.LHS.Subtract(test.RHS)
if !got.RawEquals(test.Expected) {
- t.Fatalf("Subtract returned %#v; want %#v", got, test.Expected)
+ t.Fatalf("wrong result\ngot: %#v\nwant: %#v", got, test.Expected)
}
})
}
@@ -1774,11 +2009,11 @@ func TestValueNegate(t *testing.T) {
},
{
UnknownVal(Number),
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
DynamicVal,
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
Zero.Mark(1),
@@ -1790,7 +2025,7 @@ func TestValueNegate(t *testing.T) {
t.Run(fmt.Sprintf("%#v.Negate()", test.Receiver), func(t *testing.T) {
got := test.Receiver.Negate()
if !got.RawEquals(test.Expected) {
- t.Fatalf("Negate returned %#v; want %#v", got, test.Expected)
+ t.Fatalf("wrong result\ngot: %#v\nwant: %#v", got, test.Expected)
}
})
}
@@ -1820,22 +2055,97 @@ func TestValueMultiply(t *testing.T) {
{
NumberIntVal(1),
UnknownVal(Number),
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
UnknownVal(Number),
UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
+ },
+ {
+ NumberIntVal(3),
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeLowerBound(NumberIntVal(6), true).
+ NewValue(),
+ },
+ {
+ Zero,
+ UnknownVal(Number),
+ Zero,
+ },
+ {
UnknownVal(Number),
+ Zero,
+ Zero,
+ },
+ {
+ Zero,
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), false).
+ NewValue(),
+ Zero,
+ },
+ {
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(4), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeLowerBound(NumberIntVal(8), true).
+ NewValue(),
+ },
+ {
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(3), true).
+ NumberRangeUpperBound(NumberIntVal(4), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeLowerBound(NumberIntVal(6), true).
+ NewValue(),
+ },
+ {
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(1), true).
+ NumberRangeUpperBound(NumberIntVal(2), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(2), false).
+ NumberRangeUpperBound(NumberIntVal(3), false).
+ NewValue(),
+ UnknownVal(Number).Refine().
+ NotNull().
+ NumberRangeLowerBound(NumberIntVal(2), true).
+ NumberRangeUpperBound(NumberIntVal(6), true).
+ NewValue(),
+ },
+ {
+ UnknownVal(Number).Refine().
+ NumberRangeLowerBound(NumberIntVal(1), true).
+ NumberRangeUpperBound(NumberIntVal(2), false).
+ NewValue(),
+ Zero,
+ Zero, // deduced by refinement
},
{
NumberIntVal(1),
DynamicVal,
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
DynamicVal,
DynamicVal,
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
Zero.Mark(1),
@@ -1868,7 +2178,7 @@ func TestValueMultiply(t *testing.T) {
t.Run(fmt.Sprintf("%#v.Multiply(%#v)", test.LHS, test.RHS), func(t *testing.T) {
got := test.LHS.Multiply(test.RHS)
if !got.RawEquals(test.Expected) {
- t.Fatalf("Multiply returned %#v; want %#v", got, test.Expected)
+ t.Fatalf("wrong result\ngot: %#v\nwant: %#v", got, test.Expected)
}
})
}
@@ -1908,22 +2218,22 @@ func TestValueDivide(t *testing.T) {
{
NumberIntVal(1),
UnknownVal(Number),
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
UnknownVal(Number),
UnknownVal(Number),
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
NumberIntVal(1),
DynamicVal,
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
DynamicVal,
DynamicVal,
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
Zero.Mark(1),
@@ -1946,7 +2256,7 @@ func TestValueDivide(t *testing.T) {
t.Run(fmt.Sprintf("%#v.Divide(%#v)", test.LHS, test.RHS), func(t *testing.T) {
got := test.LHS.Divide(test.RHS)
if !got.RawEquals(test.Expected) {
- t.Fatalf("Divide returned %#v; want %#v", got, test.Expected)
+ t.Fatalf("wrong result\ngot: %#v\nwant: %#v", got, test.Expected)
}
})
}
@@ -2006,22 +2316,22 @@ func TestValueModulo(t *testing.T) {
{
NumberIntVal(1),
UnknownVal(Number),
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
UnknownVal(Number),
UnknownVal(Number),
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
NumberIntVal(1),
DynamicVal,
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
DynamicVal,
DynamicVal,
- UnknownVal(Number),
+ UnknownVal(Number).RefineNotNull(),
},
{
NumberIntVal(10).Mark(1),
@@ -2086,11 +2396,11 @@ func TestValueAbsolute(t *testing.T) {
},
{
UnknownVal(Number),
- UnknownVal(Number),
+ UnknownVal(Number).Refine().NotNull().NumberRangeInclusive(Zero, UnknownVal(Number)).NewValue(),
},
{
DynamicVal,
- UnknownVal(Number),
+ UnknownVal(Number).Refine().NotNull().NumberRangeInclusive(Zero, UnknownVal(Number)).NewValue(),
},
{
NumberIntVal(-1).Mark(1),
@@ -2102,7 +2412,7 @@ func TestValueAbsolute(t *testing.T) {
t.Run(fmt.Sprintf("%#v.Absolute()", test.Receiver), func(t *testing.T) {
got := test.Receiver.Absolute()
if !got.RawEquals(test.Expected) {
- t.Fatalf("Absolute returned %#v; want %#v", got, test.Expected)
+ t.Fatalf("wrong result\ngot: %#v\nwant: %#v", got, test.Expected)
}
})
}
@@ -2336,17 +2646,17 @@ func TestValueHasIndex(t *testing.T) {
{
ListVal([]Value{StringVal("hello")}),
UnknownVal(Number),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
ListVal([]Value{StringVal("hello")}),
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
UnknownVal(List(String)),
NumberIntVal(0),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
UnknownVal(List(String)),
@@ -2386,17 +2696,17 @@ func TestValueHasIndex(t *testing.T) {
{
MapVal(map[string]Value{"greeting": StringVal("hello")}),
UnknownVal(String),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
MapVal(map[string]Value{"greeting": StringVal("hello")}),
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
UnknownVal(Map(String)),
StringVal("hello"),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
UnknownVal(Map(String)),
@@ -2441,7 +2751,7 @@ func TestValueHasIndex(t *testing.T) {
{
TupleVal([]Value{StringVal("hello")}),
UnknownVal(Number),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
UnknownVal(Tuple([]Type{String})),
@@ -2451,17 +2761,17 @@ func TestValueHasIndex(t *testing.T) {
{
TupleVal([]Value{StringVal("hello")}),
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
DynamicVal,
StringVal("hello"),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
DynamicVal,
NumberIntVal(0),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
ListVal([]Value{StringVal("hello")}).Mark(1),
@@ -2670,11 +2980,11 @@ func TestValueNot(t *testing.T) {
},
{
UnknownVal(Bool),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
True.Mark(1),
@@ -2721,32 +3031,52 @@ func TestValueAnd(t *testing.T) {
{
UnknownVal(Bool),
UnknownVal(Bool),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
True,
UnknownVal(Bool),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
UnknownVal(Bool),
True,
+ UnknownVal(Bool).RefineNotNull(),
+ },
+ {
+ False,
+ UnknownVal(Bool),
+ False,
+ },
+ {
UnknownVal(Bool),
+ False,
+ False,
},
{
DynamicVal,
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
True,
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
DynamicVal,
True,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
+ },
+ {
+ False,
+ DynamicVal,
+ False,
+ },
+ {
+ DynamicVal,
+ False,
+ False,
},
{
True.Mark(1),
@@ -2804,32 +3134,52 @@ func TestValueOr(t *testing.T) {
{
UnknownVal(Bool),
UnknownVal(Bool),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
True,
UnknownVal(Bool),
- UnknownVal(Bool),
+ True,
},
{
UnknownVal(Bool),
True,
+ True,
+ },
+ {
+ False,
UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
+ },
+ {
+ UnknownVal(Bool),
+ False,
+ UnknownVal(Bool).RefineNotNull(),
},
{
DynamicVal,
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
True,
DynamicVal,
- UnknownVal(Bool),
+ True,
},
{
DynamicVal,
True,
- UnknownVal(Bool),
+ True,
+ },
+ {
+ False,
+ DynamicVal,
+ UnknownVal(Bool).RefineNotNull(),
+ },
+ {
+ DynamicVal,
+ False,
+ UnknownVal(Bool).RefineNotNull(),
},
{
True.Mark(1),
@@ -2907,32 +3257,42 @@ func TestLessThan(t *testing.T) {
{
UnknownVal(Number),
UnknownVal(Number),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
NumberIntVal(1),
UnknownVal(Number),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
UnknownVal(Number),
NumberIntVal(1),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
+ },
+ {
+ UnknownVal(Number).Refine().NumberRangeUpperBound(Zero, true).NewValue(),
+ NumberIntVal(1),
+ True, // Deduction from the refinement
+ },
+ {
+ UnknownVal(Number).Refine().NumberRangeLowerBound(NumberIntVal(2), true).NewValue(),
+ NumberIntVal(1),
+ False, // Deduction from the refinement
},
{
DynamicVal,
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
NumberIntVal(1),
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
DynamicVal,
NumberIntVal(1),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
NumberIntVal(0).Mark(1),
@@ -3010,32 +3370,42 @@ func TestGreaterThan(t *testing.T) {
{
UnknownVal(Number),
UnknownVal(Number),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
NumberIntVal(1),
UnknownVal(Number),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
UnknownVal(Number),
NumberIntVal(1),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
+ },
+ {
+ UnknownVal(Number).Refine().NumberRangeLowerBound(NumberIntVal(2), true).NewValue(),
+ NumberIntVal(1),
+ True, // Deduction based on the refinements
+ },
+ {
+ UnknownVal(Number).Refine().NumberRangeUpperBound(NumberIntVal(0), true).NewValue(),
+ NumberIntVal(1),
+ False, // Deduction based on the refinements
},
{
DynamicVal,
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
NumberIntVal(1),
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
DynamicVal,
NumberIntVal(1),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
NumberIntVal(1).Mark(1),
@@ -3113,32 +3483,32 @@ func TestLessThanOrEqualTo(t *testing.T) {
{
UnknownVal(Number),
UnknownVal(Number),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
NumberIntVal(1),
UnknownVal(Number),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
UnknownVal(Number),
NumberIntVal(1),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
DynamicVal,
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
NumberIntVal(1),
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
DynamicVal,
NumberIntVal(1),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
NumberIntVal(0).Mark(1),
@@ -3216,32 +3586,32 @@ func TestGreaterThanOrEqualTo(t *testing.T) {
{
UnknownVal(Number),
UnknownVal(Number),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
NumberIntVal(1),
UnknownVal(Number),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
UnknownVal(Number),
NumberIntVal(1),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
DynamicVal,
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
NumberIntVal(1),
DynamicVal,
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
DynamicVal,
NumberIntVal(1),
- UnknownVal(Bool),
+ UnknownVal(Bool).RefineNotNull(),
},
{
NumberIntVal(0).Mark(1),
@@ -3299,6 +3669,30 @@ func TestValueGoString(t *testing.T) {
UnknownVal(Tuple([]Type{String, Bool})),
`cty.UnknownVal(cty.Tuple([]cty.Type{cty.String, cty.Bool}))`,
},
+ {
+ UnknownVal(String).Refine().NotNull().NewValue(),
+ `cty.UnknownVal(cty.String).RefineNotNull()`,
+ },
+ {
+ UnknownVal(String).Refine().NotNull().StringPrefix("a-").NewValue(),
+ `cty.UnknownVal(cty.String).Refine().NotNull().StringPrefixFull("a-").NewValue()`,
+ },
+ {
+ UnknownVal(String).Refine().NotNull().StringPrefix("foo").NewValue(), // The last character of the prefix gets discarded in case the next character is a combining diacritic
+ `cty.UnknownVal(cty.String).Refine().NotNull().StringPrefixFull("fo").NewValue()`,
+ },
+ {
+ UnknownVal(Bool).Refine().NotNull().NewValue(),
+ `cty.UnknownVal(cty.Bool).RefineNotNull()`,
+ },
+ {
+ UnknownVal(Number).Refine().NumberRangeInclusive(Zero, UnknownVal(Number)).NewValue(),
+ `cty.UnknownVal(cty.Number).Refine().NumberLowerBound(cty.NumberIntVal(0), true).NewValue()`,
+ },
+ {
+ UnknownVal(Number).Refine().NumberRangeInclusive(Zero, NumberIntVal(1)).NewValue(),
+ `cty.UnknownVal(cty.Number).Refine().NumberLowerBound(cty.NumberIntVal(0), true).NumberUpperBound(cty.NumberIntVal(1), true).NewValue()`,
+ },
{
StringVal(""),
diff --git a/cty/value_range.go b/cty/value_range.go
new file mode 100644
index 0000000..36f2194
--- /dev/null
+++ b/cty/value_range.go
@@ -0,0 +1,408 @@
+package cty
+
+import (
+ "fmt"
+ "math"
+ "strings"
+)
+
+// Range returns an object that offers partial information about the range
+// of the receiver.
+//
+// This is most relevant for unknown values, because it gives access to any
+// optional additional constraints on the final value (specified by the source
+// of the value using "refinements") beyond what we can assume from the value's
+// type.
+//
+// Calling Range for a known value is a little strange, but it's supported by
+// returning a [ValueRange] object that describes the exact value as closely
+// as possible. Typically a caller should work directly with the exact value
+// in that case, but some purposes might only need the level of detail
+// offered by ranges and so can share code between both known and unknown
+// values.
+func (v Value) Range() ValueRange {
+ // For an unknown value we just use its own refinements.
+ if unk, isUnk := v.v.(*unknownType); isUnk {
+ refinement := unk.refinement
+ if refinement == nil {
+ // We'll generate an unconstrained refinement, just to
+ // simplify the code in ValueRange methods which can
+ // therefore assume that there's always a refinement.
+ refinement = &refinementNullable{isNull: tristateUnknown}
+ }
+ return ValueRange{v.Type(), refinement}
+ }
+
+ if v.IsNull() {
+ // If we know a value is null then we'll just report that,
+ // since no other refinements make sense for a definitely-null value.
+ return ValueRange{
+ v.Type(),
+ &refinementNullable{isNull: tristateTrue},
+ }
+ }
+
+ // For a known value we construct synthetic refinements that match
+ // the value, just as a convenience for callers that want to share
+ // codepaths between both known and unknown values.
+ ty := v.Type()
+ var synth unknownValRefinement
+ switch {
+ case ty == String:
+ synth = &refinementString{
+ prefix: v.AsString(),
+ }
+ case ty == Number:
+ synth = &refinementNumber{
+ min: v,
+ max: v,
+ minInc: true,
+ maxInc: true,
+ }
+ case ty.IsCollectionType():
+ if lenVal := v.Length(); lenVal.IsKnown() {
+ l, _ := lenVal.AsBigFloat().Int64()
+ synth = &refinementCollection{
+ minLen: int(l),
+ maxLen: int(l),
+ }
+ } else {
+ synth = &refinementCollection{
+ minLen: 0,
+ maxLen: math.MaxInt,
+ }
+ }
+
+ default:
+ // If we don't have anything else to say then we can at least
+ // guarantee that the value isn't null.
+ synth = &refinementNullable{}
+ }
+
+ // If we get down here then the value is definitely not null
+ synth.setNull(tristateFalse)
+
+ return ValueRange{ty, synth}
+}
+
+// ValueRange offers partial information about the range of a value.
+//
+// This is primarily interesting for unknown values, because it provides access
+// to any additional known constraints (specified using "refinements") on the
+// range of the value beyond what is represented by the value's type.
+type ValueRange struct {
+ ty Type
+ raw unknownValRefinement
+}
+
+// TypeConstraint returns a type constraint describing the value's type as
+// precisely as possible with the available information.
+func (r ValueRange) TypeConstraint() Type {
+ return r.ty
+}
+
+// CouldBeNull returns true unless the value being described is definitely
+// known to represent a non-null value.
+func (r ValueRange) CouldBeNull() bool {
+ if r.raw == nil {
+ // A totally-unconstrained unknown value could be null
+ return true
+ }
+ return r.raw.null() != tristateFalse
+}
+
+// DefinitelyNotNull returns true if there are no null values in the range.
+func (r ValueRange) DefinitelyNotNull() bool {
+ if r.raw == nil {
+ // A totally-unconstrained unknown value could be null
+ return false
+ }
+ return r.raw.null() == tristateFalse
+}
+
+// NumberLowerBound returns information about the lower bound of the range of
+// a number value, or panics if the value is definitely not a number.
+//
+// If the value is nullable then the result represents the range of the number
+// only if it turns out not to be null.
+//
+// The resulting value might itself be an unknown number if there is no
+// known lower bound. In that case the "inclusive" flag is meaningless.
+func (r ValueRange) NumberLowerBound() (min Value, inclusive bool) {
+ if r.ty == DynamicPseudoType {
+ // We don't even know if this is a number yet.
+ return UnknownVal(Number), false
+ }
+ if r.ty != Number {
+ panic(fmt.Sprintf("NumberLowerBound for %#v", r.ty))
+ }
+ if rfn, ok := r.raw.(*refinementNumber); ok && rfn.min != NilVal {
+ if !rfn.min.IsKnown() {
+ return NegativeInfinity, true
+ }
+ return rfn.min, rfn.minInc
+ }
+ return NegativeInfinity, false
+}
+
+// NumberUpperBound returns information about the upper bound of the range of
+// a number value, or panics if the value is definitely not a number.
+//
+// If the value is nullable then the result represents the range of the number
+// only if it turns out not to be null.
+//
+// The resulting value might itself be an unknown number if there is no
+// known upper bound. In that case the "inclusive" flag is meaningless.
+func (r ValueRange) NumberUpperBound() (max Value, inclusive bool) {
+ if r.ty == DynamicPseudoType {
+ // We don't even know if this is a number yet.
+ return UnknownVal(Number), false
+ }
+ if r.ty != Number {
+ panic(fmt.Sprintf("NumberUpperBound for %#v", r.ty))
+ }
+ if rfn, ok := r.raw.(*refinementNumber); ok && rfn.max != NilVal {
+ if !rfn.max.IsKnown() {
+ return PositiveInfinity, true
+ }
+ return rfn.max, rfn.maxInc
+ }
+ return PositiveInfinity, false
+}
+
+// StringPrefix returns a string that is guaranteed to be the prefix of
+// the string value being described, or panics if the value is definitely not
+// a string.
+//
+// If the value is nullable then the result represents the prefix of the string
+// only if it turns out to not be null.
+//
+// If the resulting value is zero-length then the value could potentially be
+// a string but it has no known prefix.
+//
+// cty.String values always contain normalized UTF-8 sequences; the result is
+// also guaranteed to be a normalized UTF-8 sequence so the result also
+// represents the exact bytes of the string value's prefix.
+func (r ValueRange) StringPrefix() string {
+ if r.ty == DynamicPseudoType {
+ // We don't even know if this is a string yet.
+ return ""
+ }
+ if r.ty != String {
+ panic(fmt.Sprintf("StringPrefix for %#v", r.ty))
+ }
+ if rfn, ok := r.raw.(*refinementString); ok {
+ return rfn.prefix
+ }
+ return ""
+}
+
+// LengthLowerBound returns information about the lower bound of the length of
+// a collection-typed value, or panics if the value is definitely not a
+// collection.
+//
+// If the value is nullable then the result represents the range of the length
+// only if the value turns out not to be null.
+func (r ValueRange) LengthLowerBound() int {
+ if r.ty == DynamicPseudoType {
+ // We don't even know if this is a collection yet.
+ return 0
+ }
+ if !r.ty.IsCollectionType() {
+ panic(fmt.Sprintf("LengthLowerBound for %#v", r.ty))
+ }
+ if rfn, ok := r.raw.(*refinementCollection); ok {
+ return rfn.minLen
+ }
+ return 0
+}
+
+// LengthUpperBound returns information about the upper bound of the length of
+// a collection-typed value, or panics if the value is definitely not a
+// collection.
+//
+// If the value is nullable then the result represents the range of the length
+// only if the value turns out not to be null.
+//
+// The resulting value might itself be an unknown number if there is no
+// known upper bound. In that case the "inclusive" flag is meaningless.
+func (r ValueRange) LengthUpperBound() int {
+ if r.ty == DynamicPseudoType {
+ // We don't even know if this is a collection yet.
+ return math.MaxInt
+ }
+ if !r.ty.IsCollectionType() {
+ panic(fmt.Sprintf("LengthUpperBound for %#v", r.ty))
+ }
+ if rfn, ok := r.raw.(*refinementCollection); ok {
+ return rfn.maxLen
+ }
+ return math.MaxInt
+}
+
+// Includes determines whether the given value is in the receiving range.
+//
+// It can return only three possible values:
+// - [cty.True] if the range definitely includes the value
+// - [cty.False] if the range definitely does not include the value
+// - An unknown value of [cty.Bool] if there isn't enough information to decide.
+//
+// This function is not fully comprehensive: it may return an unknown value
+// in some cases where a definitive value could be computed in principle, and
+// those same situations may begin returning known values in later releases as
+// the rules are refined to be more complete. Currently the rules focus mainly
+// on answering [cty.False], because disproving membership tends to be more
+// useful than proving membership.
+func (r ValueRange) Includes(v Value) Value {
+ unknownResult := UnknownVal(Bool).RefineNotNull()
+
+ if r.raw.null() == tristateTrue {
+ if v.IsNull() {
+ return True
+ } else {
+ return False
+ }
+ }
+ if r.raw.null() == tristateFalse {
+ if v.IsNull() {
+ return False
+ }
+ // A definitely-not-null value could potentially match
+ // but we won't know until we do some more checks below.
+ }
+ // If our range includes both null and non-null values and the value is
+ // null then it's definitely in range.
+ if v.IsNull() {
+ return True
+ }
+ if len(v.Type().TestConformance(r.TypeConstraint())) != 0 {
+ // If the value doesn't conform to the type constraint then it's
+ // definitely not in the range.
+ return False
+ }
+ if v.Type() == DynamicPseudoType {
+ // If it's an unknown value of an unknown type then there's no
+ // further tests we can make.
+ return unknownResult
+ }
+
+ switch r.raw.(type) {
+ case *refinementString:
+ if v.IsKnown() {
+ prefix := r.StringPrefix()
+ got := v.AsString()
+
+ if !strings.HasPrefix(got, prefix) {
+ return False
+ }
+ }
+ case *refinementCollection:
+ lenVal := v.Length()
+ minLen := NumberIntVal(int64(r.LengthLowerBound()))
+ maxLen := NumberIntVal(int64(r.LengthUpperBound()))
+ if minOk := lenVal.GreaterThanOrEqualTo(minLen); minOk.IsKnown() && minOk.False() {
+ return False
+ }
+ if maxOk := lenVal.LessThanOrEqualTo(maxLen); maxOk.IsKnown() && maxOk.False() {
+ return False
+ }
+ case *refinementNumber:
+ minVal, minInc := r.NumberLowerBound()
+ maxVal, maxInc := r.NumberUpperBound()
+ var minOk, maxOk Value
+ if minInc {
+ minOk = v.GreaterThanOrEqualTo(minVal)
+ } else {
+ minOk = v.GreaterThan(minVal)
+ }
+ if maxInc {
+ maxOk = v.LessThanOrEqualTo(maxVal)
+ } else {
+ maxOk = v.LessThan(maxVal)
+ }
+ if minOk.IsKnown() && minOk.False() {
+ return False
+ }
+ if maxOk.IsKnown() && maxOk.False() {
+ return False
+ }
+ }
+
+ // If we fall out here then we don't have enough information to decide.
+ return unknownResult
+}
+
+// numericRangeArithmetic is a helper we use to calculate derived numeric ranges
+// for arithmetic on refined numeric values.
+//
+// op must be a monotone operation. numericRangeArithmetic adapts that operation
+// into the equivalent interval arithmetic operation.
+//
+// The result is a superset of the range of the given operation against the
+// given input ranges, if it's possible to calculate that without encountering
+// an invalid operation. Currently the result is inexact due to ignoring
+// the inclusiveness of the input bounds and just always returning inclusive
+// bounds.
+func numericRangeArithmetic(op func(a, b Value) Value, a, b ValueRange) func(*RefinementBuilder) *RefinementBuilder {
+ wrapOp := func(a, b Value) (ret Value) {
+ // Our functions have various panicking edge cases involving incompatible
+ // uses of infinities. To keep things simple here we'll catch those
+ // and just return an unconstrained number.
+ defer func() {
+ if v := recover(); v != nil {
+ ret = UnknownVal(Number)
+ }
+ }()
+ return op(a, b)
+ }
+
+ return func(builder *RefinementBuilder) *RefinementBuilder {
+ aMin, _ := a.NumberLowerBound()
+ aMax, _ := a.NumberUpperBound()
+ bMin, _ := b.NumberLowerBound()
+ bMax, _ := b.NumberUpperBound()
+
+ v1 := wrapOp(aMin, bMin)
+ v2 := wrapOp(aMin, bMax)
+ v3 := wrapOp(aMax, bMin)
+ v4 := wrapOp(aMax, bMax)
+
+ newMin := mostNumberValue(Value.LessThan, v1, v2, v3, v4)
+ newMax := mostNumberValue(Value.GreaterThan, v1, v2, v3, v4)
+
+ if isInf := newMin.Equals(NegativeInfinity); isInf.IsKnown() && isInf.False() {
+ builder = builder.NumberRangeLowerBound(newMin, true)
+ }
+ if isInf := newMax.Equals(PositiveInfinity); isInf.IsKnown() && isInf.False() {
+ builder = builder.NumberRangeUpperBound(newMax, true)
+ }
+ return builder
+ }
+}
+
+func mostNumberValue(op func(i, j Value) Value, v1 Value, vN ...Value) Value {
+ r := v1
+ for _, v := range vN {
+ more := op(v, r)
+ if !more.IsKnown() {
+ return UnknownVal(Number)
+ }
+ if more.True() {
+ r = v
+ }
+ }
+ return r
+}
+
+// definitelyNotNull is a convenient helper for the common situation of checking
+// whether a value could possibly be null.
+//
+// Returns true if the given value is either a known value that isn't null
+// or an unknown value that has been refined to exclude null values from its
+// range.
+func definitelyNotNull(v Value) bool {
+ if v.IsKnown() {
+ return !v.IsNull()
+ }
+ return v.Range().DefinitelyNotNull()
+}
diff --git a/debian/changelog b/debian/changelog
index 6073a16..f1e743a 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,11 @@
+golang-github-zclconf-go-cty (1.13.1-1) UNRELEASED; urgency=low
+
+ * New upstream release.
+ * Drop patch 0001-Skip-RFC3339-test.patch, present upstream.
+ * New upstream release.
+
+ -- Debian Janitor <janitor@jelmer.uk> Fri, 19 May 2023 21:46:46 -0000
+
golang-github-zclconf-go-cty (1.12.1-1) unstable; urgency=medium
* Team upload
diff --git a/debian/patches/0001-Skip-RFC3339-test.patch b/debian/patches/0001-Skip-RFC3339-test.patch
deleted file mode 100644
index cd4df2c..0000000
--- a/debian/patches/0001-Skip-RFC3339-test.patch
+++ /dev/null
@@ -1,27 +0,0 @@
-From: Shengjing Zhu <zhsj@debian.org>
-Date: Fri, 20 Jan 2023 15:10:38 +0800
-Subject: Skip RFC3339 test
-
-Go1.20 changes the error message to
-> cannot use "2-12-02T00:00:00Z" as year
----
- cty/function/stdlib/datetime_test.go | 6 ------
- 1 file changed, 6 deletions(-)
-
-diff --git a/cty/function/stdlib/datetime_test.go b/cty/function/stdlib/datetime_test.go
-index 7432f82..6d0e9e1 100644
---- a/cty/function/stdlib/datetime_test.go
-+++ b/cty/function/stdlib/datetime_test.go
-@@ -206,12 +206,6 @@ func TestFormatDate(t *testing.T) {
- cty.StringVal(`"2017-12-02T00:00:00Z"`),
- `not a valid RFC3339 timestamp: cannot use "\"2017-12-02T00:00:00Z\"" as year`,
- },
-- {
-- cty.StringVal(`2-12-02T00:00:00Z`),
-- // Go parser seems to be trying to parse "2-12" as a year here,
-- // producing a confusing error message.
-- `not a valid RFC3339 timestamp: cannot use "-02T00:00:00Z" as year`,
-- },
- }
- for _, test := range parseErrTests {
- t.Run(fmt.Sprintf("%s parse error", test.Timestamp.AsString()), func(t *testing.T) {
diff --git a/debian/patches/series b/debian/patches/series
index 72e1911..e69de29 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -1 +0,0 @@
-0001-Skip-RFC3339-test.patch
diff --git a/docs/concepts.md b/docs/concepts.md
index c70f80a..2f66b1d 100644
--- a/docs/concepts.md
+++ b/docs/concepts.md
@@ -75,6 +75,11 @@ promises to never produce an unknown value for an operation unless one of the
operands is itself unknown, and so applications can opt out of this additional
complexity by never providing unknown values as operands.
+At minimum an unknown value has a type constraint which describes a set of
+types that the final value could possibly have once known. In some cases we
+can refine an unknown value with additional dynamic information, using
+[Value Refinements](refinements.md).
+
## Type Equality and Type Conformance
Two types are said to be equal if they are exactly equivalent. Each type kind
diff --git a/docs/marks.md b/docs/marks.md
index 686b2ee..f136e64 100644
--- a/docs/marks.md
+++ b/docs/marks.md
@@ -106,3 +106,12 @@ unmark the whole data structure first (e.g. using `Value.UnmarkDeep`) and then
decide what to do with those marks in order to ensure that if it makes sense
to propagate them through the serialization then they will get represented
somehow.
+
+## Relationship to "Refinements"
+
+The idea of annotating a value with additional information has some overlap
+with the concept of [Refinements](refinements.md). However, the two have
+different purposes and so different design details and tradeoffs.
+
+For more details, see
+[the corresponding section in the Refinements documentation](refinements.md#relationship-to-marks).
diff --git a/docs/refinements.md b/docs/refinements.md
new file mode 100644
index 0000000..ac47451
--- /dev/null
+++ b/docs/refinements.md
@@ -0,0 +1,291 @@
+# Value Refinements
+
+_Refinements_ are dynamic annotations associated with unknown values that
+each shrink the range of possible values futher than can be represented by
+type constraint alone.
+
+When an unknown value is refined, it allows certain operations against that
+unknown value to produce a known result, and allows some operations to fail
+earlier than they would with a fully-unknown value by detecting that a valid
+result is impossible using just the refinement information.
+
+Refinements always _shrink_ the range of an unknown value, and never grow it.
+That makes it valid for some operations to ignore refinements and just treat
+an unknown value as representing any possible value of its type constraint,
+which is important to avoid burdening all downstream callers of `cty` from
+handling all refinements and from immediately adding support for new kinds of
+refinement if this model gets extended in future releases.
+
+However, note that `Value.RawEquals` _does_ take into account refinements, so
+any tests that assert against the exact final value of an operation may need
+to be updated after adopting a new version of `cty` which makes increased use
+of refinements. `Value.RawEquals` is not intended as part of the _user model_
+of `cty` and so this should not negatively impact the end-user-visible behavior
+of an application using `cty`, although of course they might benefit from
+more specific results from operations that can now take refinements into
+account.
+
+## How to refine a value
+
+You can derive a more refined value from a less refined value by using the
+`Value.Refine` method to obtain a _refinement builder_, which uses the
+builder pattern to construct a new value with one or more extra refinements.
+
+```go
+val := cty.UnknownVal(cty.String).Refine().
+ NotNull().
+ StringPrefix("https://").
+ NewValue()
+```
+
+The above snippet would produce a refined local value whose range is limited
+only to non-null strings which start with the prefix `"https://"`. This
+information can, in theory, allow `val.Equals(cty.NullVal(cty.String))` to
+return `cty.False` rather than `cty.UnknownVal(cty.Bool)`, and allow a prefix
+match against the string to return a known result.
+
+In practice not all operations against unknown values can make full use of
+unknown value refinements, but hopefully the coverage will increase over time.
+
+Only unknown values can have refinements, because known values are already
+refined by their concrete value: simple values like `cty.Zero` are constrained
+to exactly one value, while some values like `cty.ListValEmpty(cty.DynamicPseudoType)`
+represent a set of possible values -- all empty lists of any element type, in
+this case.
+
+However, the `Refine` operation _is_ also supported for known values and in that
+case acts as a self-checking assertion that the known value does actually
+meet the requirements. If you write your codepaths to unconditionally assign
+refinements regardless of whether the value is known then your code will
+self-check and raise a panic if the final known value doesn't match the
+previously-promised refinements.
+
+A similar rule applies to applying new refinements already-refined values: it's
+fine to describe a less specific refinement, which will therefore be ignored
+because it adds no new information. It's an application bug to describe a
+contradictory refinement, such as a new string prefix that doesn't match one
+previously assigned.
+
+## Value ranges
+
+The `Refine()` method described above constructs a value with refinements. To
+access the information from those refinements, use the `Value.Range` method to
+obtain a `cty.ValueRange` object, which describes a superset of all of the
+values that a particular value could have.
+
+For example, you can use `val.Range().DefinitelyNotNull()` to test whether a
+particular value is guaranteed to be non-null once it is finally known. This
+again works for both known and unknown values, so e.g.
+`cty.StringVal("foo").Range().DefinitelyNotNull()` returns `true` because
+a known, non-null string value is _definitely not null_.
+
+When writing operations that depend only on information that can be determined
+from refinements it's valid to depend exclusively on `Value.Range` and rely on
+the fact that the range of an already-known value is just a very narrow range
+that covers only what that specific value covers.
+
+The model of value ranges is imprecise, though: it's limited only to information
+we can track for unknown values through refinements. Many operations will still
+need a special codepath to handle the unknown case vs. the known case so they
+can take into account the additional detail from the exact value once known.
+
+## Available Refinements
+
+The set of possible refinement types might grow over time, but the initial set
+is focused on a narrow set of possibilities that seems likely to allow a number
+of other operations to either produce known results from unknown input or to
+rule that particular input is invalid despite not yet being known.
+
+The most notable restriction on refinements is that the available refinements
+vary depending on the type constraint of the value being refined.
+
+The least flexible case is `cty.DynamicVal` -- an unknown value of an unknown
+type -- which is the one value that cannot be refined at all and will cause
+a panic if you try. This is a pragmatic compromise for backward compatibility:
+existing callers use patterns like `val == cty.DynamicVal` to test for this
+specific special value, and any refinements of that value would make it no
+longer equal.
+
+Unknown values of built-in exact types, and also unknown values whose type
+_kind_ is constrained even if the element/attribute types are not, can at
+least be refined as being non-null, and because that is a common situation
+there is a shorthand for it which avoids using the builder pattern:
+`val.RefineNotNull()`.
+
+All other possible refinements are type-constraint-specific:
+
+* `cty.String`
+
+ For strings we can refine a known prefix of the string, which is intended
+ for situations where the string represents some microsyntax with a
+ known prefix, such as a URL of a particular known scheme.
+
+ * `.StringPrefix(string)` specifies a known prefix of the final string.
+
+ By default an unknown string has no known prefix, which is the same
+ as the prefix being the empty string.
+
+ Because `cty`'s model of strings is a sequence of Unicode grapheme
+ clusters, `.StringPrefix` will quietly disregard trailing Unicode
+ code units of the given prefix that might combine with other code
+ units to form a new combined grapheme. This is a good safe default
+ behavior for situations where the remainder of the string is under
+ end-user control and might begin with combining diacritics or
+ emoji variation sequences. Applications should not rely on the
+ details of this heuristic because it may become more precise in
+ later releases.
+
+ * `.StringPrefixFull(string)` is like `.StringPrefix` but does not trim
+ possibly-combining code units from the end of the given string.
+
+ Applications must use this with care, making sure that they control
+ the final string enough to guarantee that the subsequent additional
+ code units will never combine with any characters in the given prefix.
+
+* `cty.Number`
+
+ For numbers we can refine both the lower and upper bound of possible values,
+ with each boundary being either inclusive or exclusive.
+
+ * `.NumberRangeLowerBound(cty.Value, bool)` refines the lower bound of
+ possible values for an unknown number. The boolean argument represents
+ whether the bound is _inclusive_.
+
+ The given value must be a non-null `cty.Number` value. An unrefined
+ number effectively has a lower bound of `(cty.NegativeInfinity, true)`.
+
+ * `.NumberRangeUpperBound(cty.Value, bool)` refines the upper bound of
+ possible values for an unknown number. The boolean argument represents
+ whether the bound is _inclusive_.
+
+ The given value must be a non-null `cty.Number` value. An unrefined
+ number effectively has an upper bound of `(cty.PositiveInfinity, true)`.
+
+ * `.NumberRangeInclusive(min, max cty.Value)` is a helper wrapper around
+ the previous two methods that declares both an upper and lower bound
+ at the same time, while specifying that both are inclusive bounds.
+
+* `cty.List`, `cty.Set`, and `cty.Map` types
+
+ For all collection types we can refine the lower and upper bound of the
+ length of the collection. The boundaries on length are always inclusive
+ and are integers, because it isn't possible to have a fraction of an
+ element.
+
+ * `.CollectionLengthLowerBound(int)` refines the lower bound of possible
+ lengths for an unknown collection.
+
+ An unrefined collection effectively has a lower bound of zero, because
+ it's not possible for a collection to have a negative length.
+
+ * `.CollectionLengthUpperBound(int)` refines the upper bound of possible
+ lengths for an unknown collection.
+
+ An unrefined collection has an upper bound that matches the largest
+ valid Go slice index on the current platform, because `cty`'s
+ collections are implemented in terms of Go's collection types.
+ However, applications should typically not expose that specific value
+ to users (it's an implementation detail) and should instead present
+ the maximum value as an unconstrained length.
+
+ * `.CollectionLength(int)` is a shorthand that refines both the lower and
+ upper bounds to the same value. This is a helpful requirement to make
+ whenever possible because it will often allow the final value to be
+ a known collection with unknown elements, as described in
+ [Refinement Value Collapse](#refinement-value-collapse).
+
+Some built-in operations will automatically take into account refinements from
+their input operands and propagate them in a suitable way to the result.
+However, that is not a guarantee for all operations and so should be treated
+as a "best effort" behavior which will hopefully become more precise in future
+versions.
+
+Behaviors implemented in downstream applications, such as custom functions
+using [the function system](functions.md), might also take into account
+refinements. If they do their work using only _operation methods_ on `Value`
+then the handling of refinements might come for free. If they do work using
+_integration methods_ instead then they will need to explicitly handle
+refinements if desired. If they don't then by default the result from an
+unknown input will be a totally-unrefined unknown value, though will hopefully
+still have a useful type constraint.
+
+## Refinement Value Collapse
+
+For some kinds of refinement it's possible to constrain the range so much that
+only one possible value remains. In that case, the `.NewValue()` method of the
+refinement builder might return a known value instead of an unknown value.
+
+For example, if the lower bound and upper bound of a collection's length are
+equal then the length of the collection is effectively known. For some lengths
+of some collection kinds the refinement can collapse into a known collection
+containing unknown values. For example, an unknown list that's known to have
+exactly two values can be represented equivalently as a known list of length
+two where both elements are unknown themselves.
+
+The exact details of how refinement collapse is decided might change in future
+versions, but only in ways that can make results "more known". It would be a
+breaking change to weaken a rule to produce unknown values in more cases, so
+that kind of change would be reserved only for fixing an important bug or
+design error.
+
+## Refinements are Dynamic Only
+
+Refinements belong to unknown values rather than to type constraints, and so
+refining an unknown value does not change its type constraint.
+
+This design is a tradeoff: making the refinements dynamic and implicit means
+that it's possible to add more detailed refinements over type without making
+breaking changes to explicit type information, but the downside is that
+it isn't possible to represent refinements in any situation that is only
+aware of types.
+
+For example, it isn't currently possible to represent the idea of an unknown
+map whose elements each have a further refinement applied, because the
+refinements apply to the map itself and there are not yet any specific element
+values for the element refinements to attach to.
+
+(It would be possible in theory to allow refining an unknown collection with
+meta-refinements about its hypothetical elements, but that is not currently
+supported because it would mean that refinements would need to be resolved
+recursively and that would be considerably more complex and expensive than
+the current single-value-only refinements structure.)
+
+## Refinements Under Serialization
+
+Refinements are intentionally designed so that they only constrain the range
+of an unknown value, and never expand it. This means that it should typically
+be safe to discard refinements in situations like serialization where there
+may not be any way to represent the refinements. After decoding the unknown
+value now has a wider range but it should still be a superset of the true
+range of the value. This is an example of the general rule that no operation
+on an unknown value is _guaranteed_ to fully preserve the input refinements
+or to consider them when calculating the result.
+
+The official MessagePack serialization in particular does have some support
+for retaining approximations of refinements as part of its serialization of
+unknown values, using a MessagePack extension value. Some detail may still
+be lost under round-tripping but the output range should always be a superset
+of the input range. As long as both the serializer and deserializer are using
+the `cty/msgpack` sub-package unknown values will propagate automatically
+without any additional caller effort.
+
+## Relationship to "Marks"
+
+The idea of annotating a value with additional information has some overlap
+with the concept of [Marks](marks.md). However, the two have different purposes
+and so different design details and tradeoffs.
+
+Marks should typically be used for additional information that is independent
+of the specific type and value, such as marking a value as having come from
+a sensitive location. The marking then propagates to all results from operations
+on that value, usually without changing the behavior of that operation. In a
+sense the mark represents the _origin_ of the value rather than the value
+itself.
+
+Refinements are instead directly part of the value. By reducing the possible
+range of an unknown value placeholder, other downstream operations can in turn
+produce a more refined result, or possibly even a known result from unknown
+inputs. Refinements do not naively propagate from one value to the next, but
+some operations will use the refinements of their operands to calculate a new
+set of refiments for their result, with the rules varying on a case-by-case
+basis depending on what calculation the operation represents.
diff --git a/go.mod b/go.mod
index 8f2173c..f07ab8c 100644
--- a/go.mod
+++ b/go.mod
@@ -3,15 +3,13 @@ module github.com/zclconf/go-cty
require (
github.com/apparentlymart/go-textseg/v13 v13.0.0
github.com/google/go-cmp v0.3.1
- github.com/vmihailenco/msgpack/v4 v4.3.12
- golang.org/x/text v0.3.7
+ github.com/vmihailenco/msgpack/v5 v5.3.5
+ golang.org/x/text v0.3.8
)
require (
- github.com/golang/protobuf v1.3.4 // indirect
- github.com/vmihailenco/tagparser v0.1.1 // indirect
- golang.org/x/net v0.0.0-20200301022130-244492dfa37a // indirect
- google.golang.org/appengine v1.6.5 // indirect
+ github.com/davecgh/go-spew v1.1.1 // indirect
+ github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
)
go 1.18
diff --git a/go.sum b/go.sum
index 35938e7..cb4f7b0 100644
--- a/go.sum
+++ b/go.sum
@@ -1,31 +1,21 @@
github.com/apparentlymart/go-textseg/v13 v13.0.0 h1:Y+KvPE1NYz0xl601PVImeQfFyEy6iT90AvPUL1NNfNw=
github.com/apparentlymart/go-textseg/v13 v13.0.0/go.mod h1:ZK2fH7c4NqDTLtiYLvIkEghdlcqw7yxLeM89kiTRPUo=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
-github.com/golang/protobuf v1.3.4 h1:87PNWwrRvUSnqS4dlcBU/ftvOIBep4sYuBLlh6rX2wk=
-github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/google/go-cmp v0.3.1 h1:Xye71clBPdm5HgqGwUkwhbynsUJZhDbS20FvLhQ2izg=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
-github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
-github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
-github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
-github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
-github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
-github.com/vmihailenco/msgpack/v4 v4.3.12 h1:07s4sz9IReOgdikxLTKNbBdqDMLsjPKXwvCazn8G65U=
-github.com/vmihailenco/msgpack/v4 v4.3.12/go.mod h1:gborTTJjAo/GWTqqRjrLCn9pgNN+NXzzngzBKDPIqw4=
-github.com/vmihailenco/tagparser v0.1.1 h1:quXMXlA39OCbd2wAdTsGDlK9RkOk6Wuw+x37wVyIuWY=
-github.com/vmihailenco/tagparser v0.1.1/go.mod h1:OeAg3pn3UbLjkWt+rN9oFYB6u/cQgqMEUPoW2WPyhdI=
-golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
-golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
-golang.org/x/net v0.0.0-20200301022130-244492dfa37a h1:GuSPYbZzB5/dcLNCwLQLsg3obCJtX9IJhpXkvY7kzk0=
-golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
-golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
-golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
-golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
-golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
-google.golang.org/appengine v1.6.5 h1:tycE03LOZYQNhDpS27tcQdAzLCVMaj7QT2SXxebnpCM=
-google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
-gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
-gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0=
+github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
+github.com/vmihailenco/msgpack/v5 v5.3.5 h1:5gO0H1iULLWGhs2H5tbAHIZTV8/cYafcFOr9znI5mJU=
+github.com/vmihailenco/msgpack/v5 v5.3.5/go.mod h1:7xyJ9e+0+9SaZT0Wt1RGleJXzli6Q/V5KbhBonMG9jc=
+github.com/vmihailenco/tagparser/v2 v2.0.0 h1:y09buUbR+b5aycVFQs/g70pqKVZNBmxwAhO7/IwNM9g=
+github.com/vmihailenco/tagparser/v2 v2.0.0/go.mod h1:Wri+At7QHww0WTrCBeu4J6bNtoV6mEfg5OIWRZA9qds=
+golang.org/x/text v0.3.8 h1:nAL+RVCQ9uMn3vJZbV+MRnydTJFPf8qqY42YiA6MrqY=
+golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
More details
Historical runs
- failed: src/github.com/zclconf/go-cty/cty/set/ops.go:131:9: error: expected declaration
- success: Merged new upstream version 1.13.0
- missing-go-package: Missing Go package: github.com/apparentlymart/go-textseg/v13/textseg
- missing-go-package: Missing Go package: github.com/apparentlymart/go-textseg/v13/textseg