New Upstream Release - golang-starlark
Ready changes
Summary
Merged new upstream version: 0.0~git20240123.f864706 (was: 0.0~git20230726.7dadff3).
Diff
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index 893cc11..182f23d 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -5,15 +5,15 @@ jobs:
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
- go-version: [1.18.x, 1.19.x]
+ go-version: [1.18.x, 1.19.x, 1.20.x, 1.21.x]
runs-on: ${{ matrix.os }}
steps:
- name: Install Go
- uses: actions/setup-go@v2
+ uses: actions/setup-go@v4
with:
go-version: ${{ matrix.go-version }}
- name: Checkout code
- uses: actions/checkout@v2
+ uses: actions/checkout@v3
- name: Run Tests
shell: bash
run: 'internal/test.sh'
diff --git a/debian/changelog b/debian/changelog
index f268889..d127142 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+golang-starlark (0.0~git20240123.f864706-1) UNRELEASED; urgency=low
+
+ * New upstream snapshot.
+
+ -- Debian Janitor <janitor@jelmer.uk> Sat, 27 Jan 2024 13:10:25 -0000
+
golang-starlark (0.0~git20230726.7dadff3-2) unstable; urgency=medium
* Team upload
diff --git a/doc/spec.md b/doc/spec.md
index 6026b09..d6427e1 100644
--- a/doc/spec.md
+++ b/doc/spec.md
@@ -153,6 +153,16 @@ reproducibility is paramount, such as build tools.
* [list·insert](#list·insert)
* [list·pop](#list·pop)
* [list·remove](#list·remove)
+ * [set·add](#set·add)
+ * [set·clear](#set·clear)
+ * [set·difference](#set·difference)
+ * [set·discard](#set·discard)
+ * [set·intersection](#set·intersection)
+ * [set·issubset](#set·issubset)
+ * [set·issuperset](#set·issuperset)
+ * [set·pop](#set·pop)
+ * [set·remove](#set·remove)
+ * [set·symmetric_difference](#set·symmetric_difference)
* [set·union](#set·union)
* [string·capitalize](#string·capitalize)
* [string·codepoint_ords](#string·codepoint_ords)
@@ -966,7 +976,20 @@ Sets are instantiated by calling the built-in `set` function, which
returns a set containing all the elements of its optional argument,
which must be an iterable sequence. Sets have no literal syntax.
-The only method of a set is `union`, which is equivalent to the `|` operator.
+A set has these methods:
+
+* [`add`](#set·add)
+* [`clear`](#set·clear)
+* [`difference`](#set·difference)
+* [`discard`](#set·discard)
+* [`intersection`](#set·intersection)
+* [`issubset`](#set·issubset)
+* [`issuperset`](#set·issuperset)
+* [`pop`](#set·pop)
+* [`remove`](#set·remove)
+* [`symmetric_difference`](#set·symmetric_difference)
+* [`union`](#set·union)
+
A set used in a Boolean context is considered true if it is non-empty.
@@ -1439,7 +1462,7 @@ on the value returned by `get_filename()`.
## Value concepts
Starlark has eleven core [data types](#data-types). An application
-that embeds the Starlark intepreter may define additional types that
+that embeds the Starlark interpreter may define additional types that
behave like Starlark values. All values, whether core or
application-defined, implement a few basic behaviors:
@@ -1982,6 +2005,11 @@ which breaks several mathematical identities. For example, if `x` is
a `NaN` value, the comparisons `x < y`, `x == y`, and `x > y` all
yield false for all values of `y`.
+When used to compare two `set` objects, the `<=`, and `>=` operators will report
+whether one set is a subset or superset of another. Similarly, using `<` or `>` will
+report whether a set is a proper subset or superset of another, thus `x > y` is
+equivalent to `x >= y and x != y`.
+
Applications may define additional types that support ordered
comparison.
@@ -2032,6 +2060,8 @@ Sets
int & int # bitwise intersection (AND)
set & set # set intersection
set ^ set # set symmetric difference
+ set - set # set difference
+
Dict
dict | dict # ordered union
@@ -2102,6 +2132,7 @@ Implementations may impose a limit on the second operand of a left shift.
set([1, 2]) & set([2, 3]) # set([2])
set([1, 2]) | set([2, 3]) # set([1, 2, 3])
set([1, 2]) ^ set([2, 3]) # set([1, 3])
+set([1, 2]) - set([2, 3]) # set([1])
```
<b>Implementation note:</b>
@@ -2166,6 +2197,9 @@ which must be a tuple with exactly one component per conversion,
unless the format string contains only a single conversion, in which
case `args` itself is its operand.
+If the format string contains no conversions, the operand must be a
+`Mapping` or an empty tuple.
+
Starlark does not support the flag, width, and padding specifiers
supported by Python's `%` and other variants of C's `printf`.
@@ -3469,7 +3503,7 @@ shortest of the input sequences.
```python
zip() # []
zip(range(5)) # [(0,), (1,), (2,), (3,), (4,)]
-zip(range(5), "abc") # [(0, "a"), (1, "b"), (2, "c")]
+zip(range(5), "abc".elems()) # [(0, "a"), (1, "b"), (2, "c")]
```
## Built-in methods
@@ -3740,6 +3774,141 @@ x.remove(2) # None (x == [1, 3])
x.remove(2) # error: element not found
```
+<a id='set·add'></a>
+### set·add
+
+If `x` is not an element of set `S`, `S.add(x)` adds it to the set or fails if the set is frozen.
+If `x` already an element of the set, `add(x)` has no effect.
+
+It returns None.
+
+```python
+x = set([1, 2])
+x.add(3) # None
+x # set([1, 2, 3])
+x.add(3) # None
+x # set([1, 2, 3])
+```
+
+<a id='set·clear'></a>
+### set·clear
+
+`S.clear()` removes all items from the set or fails if the set is non-empty and frozen.
+
+It returns None.
+
+```python
+x = set([1, 2, 3])
+x.clear(2) # None
+x # set([])
+```
+
+<a id='set·difference'></a>
+### set·difference
+
+`S.difference(y)` returns a new set into which have been inserted all the elements of set S which are not in y.
+
+y can be any type of iterable (e.g. set, list, tuple).
+
+```python
+x = set([1, 2, 3])
+x.difference([3, 4, 5]) # set([1, 2])
+```
+
+<a id='set·discard'></a>
+### set·discard
+
+If `x` is an element of set `S`, `S.discard(x)` removes `x` from the set, or fails if the
+set is frozen. If `x` is not an element of the set, discard has no effect.
+
+It returns None.
+
+```python
+x = set([1, 2, 3])
+x.discard(2) # None
+x # set([1, 3])
+x.discard(2) # None
+x # set([1, 3])
+```
+
+<a id='set·intersection'></a>
+### set·intersection
+
+`S.intersection(y)` returns a new set into which have been inserted all the elements of set S which are also in y.
+
+y can be any type of iterable (e.g. set, list, tuple).
+
+```python
+x = set([1, 2, 3])
+x.intersection([3, 4, 5]) # set([3])
+```
+
+<a id='set·issubset'></a>
+### set·issubset
+
+`S.issubset(y)` returns True if all items in S are also in y, otherwise it returns False.
+
+y can be any type of iterable (e.g. set, list, tuple).
+
+```python
+x = set([1, 2])
+x.issubset([1, 2, 3]) # True
+x.issubset([1, 3, 4]) # False
+```
+
+<a id='set·issuperset'></a>
+### set·issuperset
+
+`S.issuperset(y)` returns True if all items in y are also in S, otherwise it returns False.
+
+y can be any type of iterable (e.g. set, list, tuple).
+
+```python
+x = set([1, 2, 3])
+x.issuperset([1, 2]) # True
+x.issuperset([1, 3, 4]) # False
+```
+
+<a id='set·pop'></a>
+### set·pop
+
+`S.pop()` removes the first inserted item from the set and returns it.
+
+`pop` fails if the set is empty or frozen.
+
+```python
+x = set([1, 2])
+x.pop() # 1
+x.pop() # 2
+x.pop() # error: empty set
+```
+
+<a id='set·remove'></a>
+### set·remove
+
+`S.remove(x)` removes `x` from the set and returns None.
+
+`remove` fails if the set does not contain `x` or is frozen.
+
+```python
+x = set([1, 2, 3])
+x.remove(2) # None
+x # set([1, 3])
+x.remove(2) # error: element not found
+```
+
+<a id='set·symmetric_difference'></a>
+### set·symmetric_difference
+
+`S.symmetric_difference(y)` creates a new set into which is inserted all of the items which are in S but not y, followed by all of the items which are in y but not S.
+
+y can be any type of iterable (e.g. set, list, tuple).
+
+```python
+x = set([1, 2, 3])
+x.symmetric_difference([3, 4, 5]) # set([1, 2, 4, 5])
+```
+
<a id='set·union'></a>
### set·union
diff --git a/docs/update.go b/docs/update.go
index be40427..6f88695 100644
--- a/docs/update.go
+++ b/docs/update.go
@@ -14,7 +14,6 @@ package main
import (
"bytes"
"fmt"
- "io/ioutil"
"log"
"os"
"os/exec"
@@ -51,7 +50,7 @@ func main() {
html := filepath.Join(subdir, "index.html")
if _, err := os.Stat(html); os.IsNotExist(err) {
data := strings.Replace(defaultHTML, "$PKG", pkg, -1)
- if err := ioutil.WriteFile(html, []byte(data), 0666); err != nil {
+ if err := os.WriteFile(html, []byte(data), 0666); err != nil {
log.Fatal(err)
}
log.Printf("created %s", html)
diff --git a/go.mod b/go.mod
index 719cff6..f4e3d4e 100644
--- a/go.mod
+++ b/go.mod
@@ -1,14 +1,17 @@
module go.starlark.net
-go 1.16
+go 1.18
require (
- github.com/chzyer/logex v1.1.10 // indirect
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e
- github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1 // indirect
github.com/google/go-cmp v0.5.1
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8
golang.org/x/term v0.0.0-20220526004731-065cf7ba2467
- golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
google.golang.org/protobuf v1.25.0
)
+
+require (
+ github.com/chzyer/logex v1.1.10 // indirect
+ github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1 // indirect
+ golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
+)
diff --git a/go.sum b/go.sum
index 426962b..5137682 100644
--- a/go.sum
+++ b/go.sum
@@ -43,7 +43,6 @@ golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 h1:0A+M6Uqn+Eje4kHMK80dtF3JCXC4ykBgQG4Fe06QRhQ=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20220526004731-065cf7ba2467 h1:CBpWXWQpIRjzmkkA+M7q9Fqnwd2mZr3AFqexg8YTfoM=
diff --git a/internal/chunkedfile/chunkedfile.go b/internal/chunkedfile/chunkedfile.go
index babcf1b..0751e85 100644
--- a/internal/chunkedfile/chunkedfile.go
+++ b/internal/chunkedfile/chunkedfile.go
@@ -26,7 +26,7 @@ package chunkedfile // import "go.starlark.net/internal/chunkedfile"
import (
"fmt"
- "io/ioutil"
+ "os"
"regexp"
"runtime"
"strconv"
@@ -56,7 +56,7 @@ type Reporter interface {
// by a newline so that the Go source position added by (*testing.T).Errorf
// appears on a separate line so as not to confused editors.
func Read(filename string, report Reporter) (chunks []Chunk) {
- data, err := ioutil.ReadFile(filename)
+ data, err := os.ReadFile(filename)
if err != nil {
report.Errorf("%s", err)
return
diff --git a/internal/compile/codegen_test.go b/internal/compile/codegen_test.go
index f67204f..c5954c4 100644
--- a/internal/compile/codegen_test.go
+++ b/internal/compile/codegen_test.go
@@ -64,7 +64,7 @@ func TestPlusFolding(t *testing.T) {
t.Errorf("#%d: %v", i, err)
continue
}
- got := disassemble(Expr(expr, "<expr>", locals).Toplevel)
+ got := disassemble(Expr(syntax.LegacyFileOptions(), expr, "<expr>", locals).Toplevel)
if test.want != got {
t.Errorf("expression <<%s>> generated <<%s>>, want <<%s>>",
test.src, got, test.want)
diff --git a/internal/compile/compile.go b/internal/compile/compile.go
index 888d95c..ecf689f 100644
--- a/internal/compile/compile.go
+++ b/internal/compile/compile.go
@@ -23,7 +23,6 @@
//
// Operands, logically uint32s, are encoded using little-endian 7-bit
// varints, the top bit indicating that more bytes follow.
-//
package compile // import "go.starlark.net/internal/compile"
import (
@@ -47,7 +46,7 @@ var Disassemble = false
const debug = false // make code generation verbose, for debugging the compiler
// Increment this to force recompilation of saved bytecode files.
-const Version = 13
+const Version = 14
type Opcode uint8
@@ -317,6 +316,7 @@ type Program struct {
Functions []*Funcode
Globals []Binding // for error messages and tracing
Toplevel *Funcode // module initialization function
+ Recursion bool // disable recursion check for functions in this file
}
// The type of a bytes literal value, to distinguish from text string.
@@ -486,17 +486,20 @@ func bindings(bindings []*resolve.Binding) []Binding {
}
// Expr compiles an expression to a program whose toplevel function evaluates it.
-func Expr(expr syntax.Expr, name string, locals []*resolve.Binding) *Program {
+// The options must be consistent with those used when parsing expr.
+func Expr(opts *syntax.FileOptions, expr syntax.Expr, name string, locals []*resolve.Binding) *Program {
pos := syntax.Start(expr)
stmts := []syntax.Stmt{&syntax.ReturnStmt{Result: expr}}
- return File(stmts, pos, name, locals, nil)
+ return File(opts, stmts, pos, name, locals, nil)
}
// File compiles the statements of a file into a program.
-func File(stmts []syntax.Stmt, pos syntax.Position, name string, locals, globals []*resolve.Binding) *Program {
+// The options must be consistent with those used when parsing stmts.
+func File(opts *syntax.FileOptions, stmts []syntax.Stmt, pos syntax.Position, name string, locals, globals []*resolve.Binding) *Program {
pcomp := &pcomp{
prog: &Program{
- Globals: bindings(globals),
+ Globals: bindings(globals),
+ Recursion: opts.Recursion,
},
names: make(map[string]uint32),
constants: make(map[interface{}]uint32),
diff --git a/internal/compile/serial.go b/internal/compile/serial.go
index adadabf..4d71738 100644
--- a/internal/compile/serial.go
+++ b/internal/compile/serial.go
@@ -25,6 +25,7 @@ package compile
// toplevel Funcode
// numfuncs varint
// funcs []Funcode
+// recursion varint (0 or 1)
// <strings> []byte # concatenation of all referenced strings
// EOF
//
@@ -130,6 +131,7 @@ func (prog *Program) Encode() []byte {
for _, fn := range prog.Functions {
e.function(fn)
}
+ e.int(b2i(prog.Recursion))
// Patch in the offset of the string data section.
binary.LittleEndian.PutUint32(e.p[4:8], uint32(len(e.p)))
@@ -270,6 +272,7 @@ func DecodeProgram(data []byte) (_ *Program, err error) {
for i := range funcs {
funcs[i] = d.function()
}
+ recursion := d.int() != 0
prog := &Program{
Loads: loads,
@@ -278,6 +281,7 @@ func DecodeProgram(data []byte) (_ *Program, err error) {
Globals: globals,
Functions: funcs,
Toplevel: toplevel,
+ Recursion: recursion,
}
toplevel.Prog = prog
for _, f := range funcs {
diff --git a/lib/proto/cmd/star2proto/star2proto.go b/lib/proto/cmd/star2proto/star2proto.go
index 24b5a0e..dffa603 100644
--- a/lib/proto/cmd/star2proto/star2proto.go
+++ b/lib/proto/cmd/star2proto/star2proto.go
@@ -13,7 +13,6 @@ package main
import (
"flag"
"fmt"
- "io/ioutil"
"log"
"os"
"strings"
@@ -40,8 +39,10 @@ var (
// Starlark dialect flags
func init() {
- flag.BoolVar(&resolve.AllowFloat, "fp", true, "allow floating-point numbers")
flag.BoolVar(&resolve.AllowSet, "set", resolve.AllowSet, "allow set data type")
+
+ // obsolete, no effect:
+ flag.BoolVar(&resolve.AllowFloat, "fp", true, "allow floating-point numbers")
flag.BoolVar(&resolve.AllowLambda, "lambda", resolve.AllowLambda, "allow lambda expressions")
flag.BoolVar(&resolve.AllowNestedDef, "nesteddef", resolve.AllowNestedDef, "allow nested def statements")
}
@@ -64,7 +65,7 @@ func main() {
if *descriptors != "" {
var fdset descriptorpb.FileDescriptorSet
for i, filename := range strings.Split(*descriptors, ",") {
- data, err := ioutil.ReadFile(filename)
+ data, err := os.ReadFile(filename)
if err != nil {
log.Fatalf("--descriptors[%d]: %s", i, err)
}
diff --git a/lib/time/time.go b/lib/time/time.go
index 0f78142..7dbcf36 100644
--- a/lib/time/time.go
+++ b/lib/time/time.go
@@ -6,6 +6,7 @@
package time // import "go.starlark.net/lib/time"
import (
+ "errors"
"fmt"
"sort"
"time"
@@ -18,37 +19,37 @@ import (
// Module time is a Starlark module of time-related functions and constants.
// The module defines the following functions:
//
-// from_timestamp(sec, nsec) - Converts the given Unix time corresponding to the number of seconds
-// and (optionally) nanoseconds since January 1, 1970 UTC into an object
-// of type Time. For more details, refer to https://pkg.go.dev/time#Unix.
+// from_timestamp(sec, nsec) - Converts the given Unix time corresponding to the number of seconds
+// and (optionally) nanoseconds since January 1, 1970 UTC into an object
+// of type Time. For more details, refer to https://pkg.go.dev/time#Unix.
//
-// is_valid_timezone(loc) - Reports whether loc is a valid time zone name.
+// is_valid_timezone(loc) - Reports whether loc is a valid time zone name.
//
-// now() - Returns the current local time. Applications may replace this function by a deterministic one.
+// now() - Returns the current local time. Applications may replace this function by a deterministic one.
//
-// parse_duration(d) - Parses the given duration string. For more details, refer to
-// https://pkg.go.dev/time#ParseDuration.
+// parse_duration(d) - Parses the given duration string. For more details, refer to
+// https://pkg.go.dev/time#ParseDuration.
//
-// parse_time(x, format, location) - Parses the given time string using a specific time format and location.
-// The expected arguments are a time string (mandatory), a time format
-// (optional, set to RFC3339 by default, e.g. "2021-03-22T23:20:50.52Z")
-// and a name of location (optional, set to UTC by default). For more details,
-// refer to https://pkg.go.dev/time#Parse and https://pkg.go.dev/time#ParseInLocation.
+// parse_time(x, format, location) - Parses the given time string using a specific time format and location.
+// The expected arguments are a time string (mandatory), a time format
+// (optional, set to RFC3339 by default, e.g. "2021-03-22T23:20:50.52Z")
+// and a name of location (optional, set to UTC by default). For more details,
+// refer to https://pkg.go.dev/time#Parse and https://pkg.go.dev/time#ParseInLocation.
//
-// time(year, month, day, hour, minute, second, nanosecond, location) - Returns the Time corresponding to
-// yyyy-mm-dd hh:mm:ss + nsec nanoseconds
-// in the appropriate zone for that time
-// in the given location. All the parameters
-// are optional.
-// The module also defines the following constants:
+// time(year, month, day, hour, minute, second, nanosecond, location) - Returns the Time corresponding to
+// yyyy-mm-dd hh:mm:ss + nsec nanoseconds
+// in the appropriate zone for that time
+// in the given location. All the parameters
+// are optional.
//
-// nanosecond - A duration representing one nanosecond.
-// microsecond - A duration representing one microsecond.
-// millisecond - A duration representing one millisecond.
-// second - A duration representing one second.
-// minute - A duration representing one minute.
-// hour - A duration representing one hour.
+// The module also defines the following constants:
//
+// nanosecond - A duration representing one nanosecond.
+// microsecond - A duration representing one microsecond.
+// millisecond - A duration representing one millisecond.
+// second - A duration representing one second.
+// minute - A duration representing one minute.
+// hour - A duration representing one hour.
var Module = &starlarkstruct.Module{
Name: "time",
Members: starlark.StringDict{
@@ -68,11 +69,29 @@ var Module = &starlarkstruct.Module{
},
}
-// NowFunc is a function that generates the current time. Intentionally exported
+// NowFunc is a function that reports the current time. Intentionally exported
// so that it can be overridden, for example by applications that require their
// Starlark scripts to be fully deterministic.
+//
+// Deprecated: avoid updating this global variable
+// and instead use SetNow on each thread to set its clock function.
var NowFunc = time.Now
+const contextKey = "time.now"
+
+// SetNow sets the thread's optional clock function.
+// If non-nil, it will be used in preference to NowFunc when the
+// thread requests the current time by executing a call to time.now.
+func SetNow(thread *starlark.Thread, nowFunc func() (time.Time, error)) {
+ thread.SetLocal(contextKey, nowFunc)
+}
+
+// Now returns the clock function previously associated with this thread.
+func Now(thread *starlark.Thread) func() (time.Time, error) {
+ nowFunc, _ := thread.Local(contextKey).(func() (time.Time, error))
+ return nowFunc
+}
+
func parseDuration(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) {
var d Duration
err := starlark.UnpackPositionalArgs("parse_duration", args, kwargs, 1, &d)
@@ -129,7 +148,19 @@ func fromTimestamp(thread *starlark.Thread, _ *starlark.Builtin, args starlark.T
}
func now(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) {
- return Time(NowFunc()), nil
+ nowErrFunc := Now(thread)
+ if nowErrFunc != nil {
+ t, err := nowErrFunc()
+ if err != nil {
+ return nil, err
+ }
+ return Time(t), nil
+ }
+ nowFunc := NowFunc
+ if nowFunc == nil {
+ return nil, errors.New("time.now() is not available")
+ }
+ return Time(nowFunc()), nil
}
// Duration is a Starlark representation of a duration.
@@ -222,14 +253,15 @@ func (d Duration) Cmp(v starlark.Value, depth int) (int, error) {
// Binary implements binary operators, which satisfies the starlark.HasBinary
// interface. operators:
-// duration + duration = duration
-// duration + time = time
-// duration - duration = duration
-// duration / duration = float
-// duration / int = duration
-// duration / float = duration
-// duration // duration = int
-// duration * int = duration
+//
+// duration + duration = duration
+// duration + time = time
+// duration - duration = duration
+// duration / duration = float
+// duration / int = duration
+// duration / float = duration
+// duration // duration = int
+// duration * int = duration
func (d Duration) Binary(op syntax.Token, y starlark.Value, side starlark.Side) (starlark.Value, error) {
x := time.Duration(d)
@@ -329,8 +361,9 @@ func newTime(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple,
}
// String returns the time formatted using the format string
+//
// "2006-01-02 15:04:05.999999999 -0700 MST".
-func (t Time) String() string { return time.Time(t).String() }
+func (t Time) String() string { return time.Time(t).Format("2006-01-02 15:04:05.999999999 -0700 MST") }
// Type returns "time.time".
func (t Time) Type() string { return "time.time" }
@@ -406,9 +439,10 @@ func (t Time) Cmp(yV starlark.Value, depth int) (int, error) {
// Binary implements binary operators, which satisfies the starlark.HasBinary
// interface
-// time + duration = time
-// time - duration = time
-// time - time = duration
+//
+// time + duration = time
+// time - duration = time
+// time - time = duration
func (t Time) Binary(op syntax.Token, y starlark.Value, side starlark.Side) (starlark.Value, error) {
x := time.Time(t)
diff --git a/lib/time/time_test.go b/lib/time/time_test.go
new file mode 100644
index 0000000..b799953
--- /dev/null
+++ b/lib/time/time_test.go
@@ -0,0 +1,82 @@
+package time
+
+import (
+ "errors"
+ "testing"
+ "time"
+
+ "go.starlark.net/starlark"
+)
+
+func TestPerThreadNowReturnsCorrectTime(t *testing.T) {
+ th := &starlark.Thread{}
+ date := time.Date(1, 2, 3, 4, 5, 6, 7, time.UTC)
+ SetNow(th, func() (time.Time, error) {
+ return date, nil
+ })
+
+ res, err := starlark.Call(th, Module.Members["now"], nil, nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ retTime := time.Time(res.(Time))
+
+ if !retTime.Equal(date) {
+ t.Fatal("Expected time to be equal", retTime, date)
+ }
+}
+
+func TestPerThreadNowReturnsError(t *testing.T) {
+ th := &starlark.Thread{}
+ e := errors.New("no time")
+ SetNow(th, func() (time.Time, error) {
+ return time.Time{}, e
+ })
+
+ _, err := starlark.Call(th, Module.Members["now"], nil, nil)
+ if !errors.Is(err, e) {
+ t.Fatal("Expected equal error", e, err)
+ }
+}
+
+func TestGlobalNowReturnsCorrectTime(t *testing.T) {
+ th := &starlark.Thread{}
+
+ oldNow := NowFunc
+ defer func() {
+ NowFunc = oldNow
+ }()
+
+ date := time.Date(1, 2, 3, 4, 5, 6, 7, time.UTC)
+ NowFunc = func() time.Time {
+ return date
+ }
+
+ res, err := starlark.Call(th, Module.Members["now"], nil, nil)
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ retTime := time.Time(res.(Time))
+
+ if !retTime.Equal(date) {
+ t.Fatal("Expected time to be equal", retTime, date)
+ }
+}
+
+func TestGlobalNowReturnsErrorWhenNil(t *testing.T) {
+ th := &starlark.Thread{}
+
+ oldNow := NowFunc
+ defer func() {
+ NowFunc = oldNow
+ }()
+
+ NowFunc = nil
+
+ _, err := starlark.Call(th, Module.Members["now"], nil, nil)
+ if err == nil {
+ t.Fatal("Expected to get an error")
+ }
+}
diff --git a/repl/repl.go b/repl/repl.go
index 94f8947..6bb7f56 100644
--- a/repl/repl.go
+++ b/repl/repl.go
@@ -20,21 +20,25 @@ import (
"os/signal"
"github.com/chzyer/readline"
- "go.starlark.net/resolve"
"go.starlark.net/starlark"
"go.starlark.net/syntax"
)
var interrupted = make(chan os.Signal, 1)
-// REPL executes a read, eval, print loop.
+// REPL calls [REPLOptions] using [syntax.LegacyFileOptions].
+// Deprecated: relies on legacy global variables.
+func REPL(thread *starlark.Thread, globals starlark.StringDict) {
+ REPLOptions(syntax.LegacyFileOptions(), thread, globals)
+}
+
+// REPLOptions executes a read, eval, print loop.
//
// Before evaluating each expression, it sets the Starlark thread local
// variable named "context" to a context.Context that is cancelled by a
// SIGINT (Control-C). Client-supplied global functions may use this
// context to make long-running operations interruptable.
-//
-func REPL(thread *starlark.Thread, globals starlark.StringDict) {
+func REPLOptions(opts *syntax.FileOptions, thread *starlark.Thread, globals starlark.StringDict) {
signal.Notify(interrupted, os.Interrupt)
defer signal.Stop(interrupted)
@@ -45,7 +49,7 @@ func REPL(thread *starlark.Thread, globals starlark.StringDict) {
}
defer rl.Close()
for {
- if err := rep(rl, thread, globals); err != nil {
+ if err := rep(opts, rl, thread, globals); err != nil {
if err == readline.ErrInterrupt {
fmt.Println(err)
continue
@@ -59,7 +63,7 @@ func REPL(thread *starlark.Thread, globals starlark.StringDict) {
//
// It returns an error (possibly readline.ErrInterrupt)
// only if readline failed. Starlark errors are printed.
-func rep(rl *readline.Instance, thread *starlark.Thread, globals starlark.StringDict) error {
+func rep(opts *syntax.FileOptions, rl *readline.Instance, thread *starlark.Thread, globals starlark.StringDict) error {
// Each item gets its own context,
// which is cancelled by a SIGINT.
//
@@ -93,8 +97,14 @@ func rep(rl *readline.Instance, thread *starlark.Thread, globals starlark.String
return []byte(line + "\n"), nil
}
+ // Treat load bindings as global (like they used to be) in the REPL.
+ // Fixes github.com/google/starlark-go/issues/224.
+ opts2 := *opts
+ opts2.LoadBindsGlobally = true
+ opts = &opts2
+
// parse
- f, err := syntax.ParseCompoundStmt("<stdin>", readline)
+ f, err := opts.ParseCompoundStmt("<stdin>", readline)
if err != nil {
if eof {
return io.EOF
@@ -103,16 +113,9 @@ func rep(rl *readline.Instance, thread *starlark.Thread, globals starlark.String
return nil
}
- // Treat load bindings as global (like they used to be) in the REPL.
- // This is a workaround for github.com/google/starlark-go/issues/224.
- // TODO(adonovan): not safe wrt concurrent interpreters.
- // Come up with a more principled solution (or plumb options everywhere).
- defer func(prev bool) { resolve.LoadBindsGlobally = prev }(resolve.LoadBindsGlobally)
- resolve.LoadBindsGlobally = true
-
if expr := soleExpr(f); expr != nil {
// eval
- v, err := starlark.EvalExpr(thread, expr, globals)
+ v, err := starlark.EvalExprOptions(f.Options, thread, expr, globals)
if err != nil {
PrintError(err)
return nil
@@ -149,10 +152,16 @@ func PrintError(err error) {
}
}
-// MakeLoad returns a simple sequential implementation of module loading
-// suitable for use in the REPL.
-// Each function returned by MakeLoad accesses a distinct private cache.
+// MakeLoad calls [MakeLoadOptions] using [syntax.LegacyFileOptions].
+// Deprecated: relies on legacy global variables.
func MakeLoad() func(thread *starlark.Thread, module string) (starlark.StringDict, error) {
+ return MakeLoadOptions(syntax.LegacyFileOptions())
+}
+
+// MakeLoadOptions returns a simple sequential implementation of module loading
+// suitable for use in the REPL.
+// Each function returned by MakeLoadOptions accesses a distinct private cache.
+func MakeLoadOptions(opts *syntax.FileOptions) func(thread *starlark.Thread, module string) (starlark.StringDict, error) {
type entry struct {
globals starlark.StringDict
err error
@@ -173,7 +182,7 @@ func MakeLoad() func(thread *starlark.Thread, module string) (starlark.StringDic
// Load it.
thread := &starlark.Thread{Name: "exec " + module, Load: thread.Load}
- globals, err := starlark.ExecFile(thread, module, nil, nil)
+ globals, err := starlark.ExecFileOptions(opts, thread, module, nil, nil)
e = &entry{globals, err}
// Update the cache.
diff --git a/resolve/binding.go b/resolve/binding.go
index 6b99f4b..8507e64 100644
--- a/resolve/binding.go
+++ b/resolve/binding.go
@@ -10,7 +10,7 @@ import "go.starlark.net/syntax"
// We cannot guarantee API stability for these types
// as they are closely tied to the implementation.
-// A Binding contains resolver information about an identifer.
+// A Binding contains resolver information about an identifier.
// The resolver populates the Binding field of each syntax.Identifier.
// The Binding ties together all identifiers that denote the same variable.
type Binding struct {
diff --git a/resolve/resolve.go b/resolve/resolve.go
index 09b9acd..c576a6b 100644
--- a/resolve/resolve.go
+++ b/resolve/resolve.go
@@ -97,6 +97,9 @@ const doesnt = "this Starlark dialect does not "
// global options
// These features are either not standard Starlark (yet), or deprecated
// features of the BUILD language, so we put them behind flags.
+//
+// Deprecated: use an explicit [syntax.FileOptions] argument instead,
+// as it avoids all the usual problems of global variables.
var (
AllowSet = false // allow the 'set' built-in
AllowGlobalReassign = false // allow reassignment to top-level names; also, allow if/for/while at top-level
@@ -130,7 +133,7 @@ func File(file *syntax.File, isPredeclared, isUniversal func(name string) bool)
// REPLChunk is a generalization of the File function that supports a
// non-empty initial global block, as occurs in a REPL.
func REPLChunk(file *syntax.File, isGlobal, isPredeclared, isUniversal func(name string) bool) error {
- r := newResolver(isGlobal, isPredeclared, isUniversal)
+ r := newResolver(file.Options, isGlobal, isPredeclared, isUniversal)
r.stmts(file.Stmts)
r.env.resolveLocalUses()
@@ -151,12 +154,18 @@ func REPLChunk(file *syntax.File, isGlobal, isPredeclared, isUniversal func(name
return nil
}
-// Expr resolves the specified expression.
+// Expr calls [ExprOptions] using [syntax.LegacyFileOptions].
+// Deprecated: relies on legacy global variables.
+func Expr(expr syntax.Expr, isPredeclared, isUniversal func(name string) bool) ([]*Binding, error) {
+ return ExprOptions(syntax.LegacyFileOptions(), expr, isPredeclared, isUniversal)
+}
+
+// ExprOptions resolves the specified expression.
// It returns the local variables bound within the expression.
//
-// The isPredeclared and isUniversal predicates behave as for the File function.
-func Expr(expr syntax.Expr, isPredeclared, isUniversal func(name string) bool) ([]*Binding, error) {
- r := newResolver(nil, isPredeclared, isUniversal)
+// The isPredeclared and isUniversal predicates behave as for the File function
+func ExprOptions(opts *syntax.FileOptions, expr syntax.Expr, isPredeclared, isUniversal func(name string) bool) ([]*Binding, error) {
+ r := newResolver(opts, nil, isPredeclared, isUniversal)
r.expr(expr)
r.env.resolveLocalUses()
r.resolveNonLocalUses(r.env) // globals & universals
@@ -179,9 +188,10 @@ type Error struct {
func (e Error) Error() string { return e.Pos.String() + ": " + e.Msg }
-func newResolver(isGlobal, isPredeclared, isUniversal func(name string) bool) *resolver {
+func newResolver(options *syntax.FileOptions, isGlobal, isPredeclared, isUniversal func(name string) bool) *resolver {
file := new(block)
return &resolver{
+ options: options,
file: file,
env: file,
isGlobal: isGlobal,
@@ -193,6 +203,8 @@ func newResolver(isGlobal, isPredeclared, isUniversal func(name string) bool) *r
}
type resolver struct {
+ options *syntax.FileOptions
+
// env is the current local environment:
// a linked list of blocks, innermost first.
// The tail of the list is the file block.
@@ -314,7 +326,7 @@ func (r *resolver) bind(id *syntax.Ident) bool {
r.moduleGlobals = append(r.moduleGlobals, bind)
}
}
- if ok && !AllowGlobalReassign {
+ if ok && !r.options.GlobalReassign {
r.errorf(id.NamePos, "cannot reassign %s %s declared at %s",
bind.Scope, id.Name, bind.First.NamePos)
}
@@ -382,7 +394,7 @@ func (r *resolver) use(id *syntax.Ident) {
// We will piggyback support for the legacy semantics on the
// AllowGlobalReassign flag, which is loosely related and also
// required for Bazel.
- if AllowGlobalReassign && r.env == r.file {
+ if r.options.GlobalReassign && r.env == r.file {
r.useToplevel(use)
return
}
@@ -420,7 +432,7 @@ func (r *resolver) useToplevel(use use) (bind *Binding) {
r.predeclared[id.Name] = bind // save it
} else if r.isUniversal(id.Name) {
// use of universal name
- if !AllowSet && id.Name == "set" {
+ if !r.options.Set && id.Name == "set" {
r.errorf(id.NamePos, doesnt+"support sets")
}
bind = &Binding{Scope: Universal}
@@ -493,7 +505,7 @@ func (r *resolver) stmt(stmt syntax.Stmt) {
}
case *syntax.IfStmt:
- if !AllowGlobalReassign && r.container().function == nil {
+ if !r.options.TopLevelControl && r.container().function == nil {
r.errorf(stmt.If, "if statement not within a function")
}
r.expr(stmt.Cond)
@@ -519,7 +531,7 @@ func (r *resolver) stmt(stmt syntax.Stmt) {
r.function(fn, stmt.Def)
case *syntax.ForStmt:
- if !AllowGlobalReassign && r.container().function == nil {
+ if !r.options.TopLevelControl && r.container().function == nil {
r.errorf(stmt.For, "for loop not within a function")
}
r.expr(stmt.X)
@@ -530,10 +542,10 @@ func (r *resolver) stmt(stmt syntax.Stmt) {
r.loops--
case *syntax.WhileStmt:
- if !AllowRecursion {
+ if !r.options.While {
r.errorf(stmt.While, doesnt+"support while loops")
}
- if !AllowGlobalReassign && r.container().function == nil {
+ if !r.options.TopLevelControl && r.container().function == nil {
r.errorf(stmt.While, "while loop not within a function")
}
r.expr(stmt.Cond)
@@ -569,9 +581,9 @@ func (r *resolver) stmt(stmt syntax.Stmt) {
}
id := stmt.To[i]
- if LoadBindsGlobally {
+ if r.options.LoadBindsGlobally {
r.bind(id)
- } else if r.bindLocal(id) && !AllowGlobalReassign {
+ } else if r.bindLocal(id) && !r.options.GlobalReassign {
// "Global" in AllowGlobalReassign is a misnomer for "toplevel".
// Sadly we can't report the previous declaration
// as id.Binding may not be set yet.
diff --git a/resolve/resolve_test.go b/resolve/resolve_test.go
index 23bee21..31bba18 100644
--- a/resolve/resolve_test.go
+++ b/resolve/resolve_test.go
@@ -14,11 +14,16 @@ import (
"go.starlark.net/syntax"
)
-func setOptions(src string) {
- resolve.AllowGlobalReassign = option(src, "globalreassign")
- resolve.AllowRecursion = option(src, "recursion")
- resolve.AllowSet = option(src, "set")
- resolve.LoadBindsGlobally = option(src, "loadbindsglobally")
+// A test may enable non-standard options by containing (e.g.) "option:recursion".
+func getOptions(src string) *syntax.FileOptions {
+ return &syntax.FileOptions{
+ Set: option(src, "set"),
+ While: option(src, "while"),
+ TopLevelControl: option(src, "toplevelcontrol"),
+ GlobalReassign: option(src, "globalreassign"),
+ LoadBindsGlobally: option(src, "loadbindsglobally"),
+ Recursion: option(src, "recursion"),
+ }
}
func option(chunk, name string) bool {
@@ -26,18 +31,17 @@ func option(chunk, name string) bool {
}
func TestResolve(t *testing.T) {
- defer setOptions("")
filename := starlarktest.DataFile("resolve", "testdata/resolve.star")
for _, chunk := range chunkedfile.Read(filename, t) {
- f, err := syntax.Parse(filename, chunk.Source, 0)
+ // A chunk may set options by containing e.g. "option:recursion".
+ opts := getOptions(chunk.Source)
+
+ f, err := opts.Parse(filename, chunk.Source, 0)
if err != nil {
t.Error(err)
continue
}
- // A chunk may set options by containing e.g. "option:recursion".
- setOptions(chunk.Source)
-
if err := resolve.File(f, isPredeclared, isUniversal); err != nil {
for _, err := range err.(resolve.ErrorList) {
chunk.GotError(int(err.Pos.Line), err.Msg)
diff --git a/resolve/testdata/resolve.star b/resolve/testdata/resolve.star
index 4fca831..cb6385a 100644
--- a/resolve/testdata/resolve.star
+++ b/resolve/testdata/resolve.star
@@ -143,17 +143,17 @@ load("foo",
_e="f") # ok
---
-# option:globalreassign
+# option:toplevelcontrol
if M:
load("foo", "bar") ### "load statement within a conditional"
---
-# option:globalreassign
+# option:toplevelcontrol
for x in M:
load("foo", "bar") ### "load statement within a loop"
---
-# option:recursion option:globalreassign
+# option:toplevelcontrol option:while
while M:
load("foo", "bar") ### "load statement within a loop"
@@ -173,7 +173,7 @@ if x: ### "if statement not within a function"
pass
---
-# option:globalreassign
+# option:toplevelcontrol
for x in "abc": # ok
pass
@@ -189,7 +189,7 @@ def f():
pass
---
-# option:recursion
+# option:while
def f():
while U: # ok
@@ -199,7 +199,7 @@ while U: ### "while loop not within a function"
pass
---
-# option:globalreassign option:recursion
+# option:toplevelcontrol option:while
while U: # ok
pass
diff --git a/starlark/bench_test.go b/starlark/bench_test.go
index e860df7..4ff0789 100644
--- a/starlark/bench_test.go
+++ b/starlark/bench_test.go
@@ -7,7 +7,7 @@ package starlark_test
import (
"bytes"
"fmt"
- "io/ioutil"
+ "os"
"path/filepath"
"strings"
"testing"
@@ -18,8 +18,6 @@ import (
)
func BenchmarkStarlark(b *testing.B) {
- defer setOptions("")
-
starlark.Universe["json"] = json.Module
testdata := starlarktest.DataFile("starlark", ".")
@@ -31,15 +29,15 @@ func BenchmarkStarlark(b *testing.B) {
filename := filepath.Join(testdata, file)
- src, err := ioutil.ReadFile(filename)
+ src, err := os.ReadFile(filename)
if err != nil {
b.Error(err)
continue
}
- setOptions(string(src))
+ opts := getOptions(string(src))
// Evaluate the file once.
- globals, err := starlark.ExecFile(thread, filename, src, nil)
+ globals, err := starlark.ExecFileOptions(opts, thread, filename, src, nil)
if err != nil {
reportEvalError(b, err)
}
@@ -63,9 +61,9 @@ func BenchmarkStarlark(b *testing.B) {
// It provides b.n, the number of iterations that must be executed by the function,
// which is typically of the form:
//
-// def bench_foo(b):
-// for _ in range(b.n):
-// ...work...
+// def bench_foo(b):
+// for _ in range(b.n):
+// ...work...
//
// It also provides stop, start, and restart methods to stop the clock in case
// there is significant set-up work that should not count against the measured
@@ -128,7 +126,7 @@ func BenchmarkProgram(b *testing.B) {
b.Run("read", func(b *testing.B) {
for i := 0; i < b.N; i++ {
var err error
- src, err = ioutil.ReadFile(filename)
+ src, err = os.ReadFile(filename)
if err != nil {
b.Fatal(err)
}
diff --git a/starlark/eval.go b/starlark/eval.go
index 949cb93..6c11bc4 100644
--- a/starlark/eval.go
+++ b/starlark/eval.go
@@ -7,7 +7,6 @@ package starlark
import (
"fmt"
"io"
- "io/ioutil"
"log"
"math/big"
"sort"
@@ -325,7 +324,13 @@ func (prog *Program) Write(out io.Writer) error {
return err
}
-// ExecFile parses, resolves, and executes a Starlark file in the
+// ExecFile calls [ExecFileOptions] using [syntax.LegacyFileOptions].
+// Deprecated: relies on legacy global variables.
+func ExecFile(thread *Thread, filename string, src interface{}, predeclared StringDict) (StringDict, error) {
+ return ExecFileOptions(syntax.LegacyFileOptions(), thread, filename, src, predeclared)
+}
+
+// ExecFileOptions parses, resolves, and executes a Starlark file in the
// specified global environment, which may be modified during execution.
//
// Thread is the state associated with the Starlark thread.
@@ -340,11 +345,11 @@ func (prog *Program) Write(out io.Writer) error {
// Execution does not modify this dictionary, though it may mutate
// its values.
//
-// If ExecFile fails during evaluation, it returns an *EvalError
+// If ExecFileOptions fails during evaluation, it returns an *EvalError
// containing a backtrace.
-func ExecFile(thread *Thread, filename string, src interface{}, predeclared StringDict) (StringDict, error) {
+func ExecFileOptions(opts *syntax.FileOptions, thread *Thread, filename string, src interface{}, predeclared StringDict) (StringDict, error) {
// Parse, resolve, and compile a Starlark source file.
- _, mod, err := SourceProgram(filename, src, predeclared.Has)
+ _, mod, err := SourceProgramOptions(opts, filename, src, predeclared.Has)
if err != nil {
return nil, err
}
@@ -354,7 +359,13 @@ func ExecFile(thread *Thread, filename string, src interface{}, predeclared Stri
return g, err
}
-// SourceProgram produces a new program by parsing, resolving,
+// SourceProgram calls [SourceProgramOptions] using [syntax.LegacyFileOptions].
+// Deprecated: relies on legacy global variables.
+func SourceProgram(filename string, src interface{}, isPredeclared func(string) bool) (*syntax.File, *Program, error) {
+ return SourceProgramOptions(syntax.LegacyFileOptions(), filename, src, isPredeclared)
+}
+
+// SourceProgramOptions produces a new program by parsing, resolving,
// and compiling a Starlark source file.
// On success, it returns the parsed file and the compiled program.
// The filename and src parameters are as for syntax.Parse.
@@ -363,8 +374,8 @@ func ExecFile(thread *Thread, filename string, src interface{}, predeclared Stri
// a pre-declared identifier of the current module.
// Its typical value is predeclared.Has,
// where predeclared is a StringDict of pre-declared values.
-func SourceProgram(filename string, src interface{}, isPredeclared func(string) bool) (*syntax.File, *Program, error) {
- f, err := syntax.Parse(filename, src, 0)
+func SourceProgramOptions(opts *syntax.FileOptions, filename string, src interface{}, isPredeclared func(string) bool) (*syntax.File, *Program, error) {
+ f, err := opts.Parse(filename, src, 0)
if err != nil {
return nil, nil, err
}
@@ -396,7 +407,7 @@ func FileProgram(f *syntax.File, isPredeclared func(string) bool) (*Program, err
}
module := f.Module.(*resolve.Module)
- compiled := compile.File(f.Stmts, pos, "<toplevel>", module.Locals, module.Globals)
+ compiled := compile.File(f.Options, f.Stmts, pos, "<toplevel>", module.Locals, module.Globals)
return &Program{compiled}, nil
}
@@ -404,7 +415,7 @@ func FileProgram(f *syntax.File, isPredeclared func(string) bool) (*Program, err
// CompiledProgram produces a new program from the representation
// of a compiled program previously saved by Program.Write.
func CompiledProgram(in io.Reader) (*Program, error) {
- data, err := ioutil.ReadAll(in)
+ data, err := io.ReadAll(in)
if err != nil {
return nil, err
}
@@ -453,7 +464,7 @@ func ExecREPLChunk(f *syntax.File, thread *Thread, globals StringDict) error {
}
module := f.Module.(*resolve.Module)
- compiled := compile.File(f.Stmts, pos, "<toplevel>", module.Locals, module.Globals)
+ compiled := compile.File(f.Options, f.Stmts, pos, "<toplevel>", module.Locals, module.Globals)
prog := &Program{compiled}
// -- variant of Program.Init --
@@ -512,7 +523,13 @@ func makeToplevelFunction(prog *compile.Program, predeclared StringDict) *Functi
}
}
-// Eval parses, resolves, and evaluates an expression within the
+// Eval calls [EvalOptions] using [syntax.LegacyFileOptions].
+// Deprecated: relies on legacy global variables.
+func Eval(thread *Thread, filename string, src interface{}, env StringDict) (Value, error) {
+ return EvalOptions(syntax.LegacyFileOptions(), thread, filename, src, env)
+}
+
+// EvalOptions parses, resolves, and evaluates an expression within the
// specified (predeclared) environment.
//
// Evaluation cannot mutate the environment dictionary itself,
@@ -520,58 +537,71 @@ func makeToplevelFunction(prog *compile.Program, predeclared StringDict) *Functi
//
// The filename and src parameters are as for syntax.Parse.
//
-// If Eval fails during evaluation, it returns an *EvalError
+// If EvalOptions fails during evaluation, it returns an *EvalError
// containing a backtrace.
-func Eval(thread *Thread, filename string, src interface{}, env StringDict) (Value, error) {
- expr, err := syntax.ParseExpr(filename, src, 0)
+func EvalOptions(opts *syntax.FileOptions, thread *Thread, filename string, src interface{}, env StringDict) (Value, error) {
+ expr, err := opts.ParseExpr(filename, src, 0)
if err != nil {
return nil, err
}
- f, err := makeExprFunc(expr, env)
+ f, err := makeExprFunc(opts, expr, env)
if err != nil {
return nil, err
}
return Call(thread, f, nil, nil)
}
-// EvalExpr resolves and evaluates an expression within the
+// EvalExpr calls [EvalExprOptions] using [syntax.LegacyFileOptions].
+// Deprecated: relies on legacy global variables.
+func EvalExpr(thread *Thread, expr syntax.Expr, env StringDict) (Value, error) {
+ return EvalExprOptions(syntax.LegacyFileOptions(), thread, expr, env)
+}
+
+// EvalExprOptions resolves and evaluates an expression within the
// specified (predeclared) environment.
// Evaluating a comma-separated list of expressions yields a tuple value.
//
// Resolving an expression mutates it.
-// Do not call EvalExpr more than once for the same expression.
+// Do not call EvalExprOptions more than once for the same expression.
//
// Evaluation cannot mutate the environment dictionary itself,
// though it may modify variables reachable from the dictionary.
//
-// If Eval fails during evaluation, it returns an *EvalError
+// If EvalExprOptions fails during evaluation, it returns an *EvalError
// containing a backtrace.
-func EvalExpr(thread *Thread, expr syntax.Expr, env StringDict) (Value, error) {
- fn, err := makeExprFunc(expr, env)
+func EvalExprOptions(opts *syntax.FileOptions, thread *Thread, expr syntax.Expr, env StringDict) (Value, error) {
+ fn, err := makeExprFunc(opts, expr, env)
if err != nil {
return nil, err
}
return Call(thread, fn, nil, nil)
}
+// ExprFunc calls [ExprFuncOptions] using [syntax.LegacyFileOptions].
+// Deprecated: relies on legacy global variables.
+func ExprFunc(filename string, src interface{}, env StringDict) (*Function, error) {
+ return ExprFuncOptions(syntax.LegacyFileOptions(), filename, src, env)
+}
+
// ExprFunc returns a no-argument function
// that evaluates the expression whose source is src.
-func ExprFunc(filename string, src interface{}, env StringDict) (*Function, error) {
- expr, err := syntax.ParseExpr(filename, src, 0)
+func ExprFuncOptions(options *syntax.FileOptions, filename string, src interface{}, env StringDict) (*Function, error) {
+ expr, err := options.ParseExpr(filename, src, 0)
if err != nil {
return nil, err
}
- return makeExprFunc(expr, env)
+ return makeExprFunc(options, expr, env)
}
// makeExprFunc returns a no-argument function whose body is expr.
-func makeExprFunc(expr syntax.Expr, env StringDict) (*Function, error) {
- locals, err := resolve.Expr(expr, env.Has, Universe.Has)
+// The options must be consistent with those used when parsing expr.
+func makeExprFunc(opts *syntax.FileOptions, expr syntax.Expr, env StringDict) (*Function, error) {
+ locals, err := resolve.ExprOptions(opts, expr, env.Has, Universe.Has)
if err != nil {
return nil, err
}
- return makeToplevelFunction(compile.Expr(expr, "<expr>", locals), env), nil
+ return makeToplevelFunction(compile.Expr(opts, expr, "<expr>", locals), env), nil
}
// The following functions are primitive operations of the byte code interpreter.
@@ -795,6 +825,12 @@ func Binary(op syntax.Token, x, y Value) (Value, error) {
}
return x - yf, nil
}
+ case *Set: // difference
+ if y, ok := y.(*Set); ok {
+ iter := y.Iterate()
+ defer iter.Done()
+ return x.Difference(iter)
+ }
}
case syntax.STAR:
@@ -1066,17 +1102,9 @@ func Binary(op syntax.Token, x, y Value) (Value, error) {
}
case *Set: // intersection
if y, ok := y.(*Set); ok {
- set := new(Set)
- if x.Len() > y.Len() {
- x, y = y, x // opt: range over smaller set
- }
- for xe := x.ht.head; xe != nil; xe = xe.next {
- // Has, Insert cannot fail here.
- if found, _ := y.Has(xe.key); found {
- set.Insert(xe.key)
- }
- }
- return set, nil
+ iter := y.Iterate()
+ defer iter.Done()
+ return x.Intersection(iter)
}
}
@@ -1088,18 +1116,9 @@ func Binary(op syntax.Token, x, y Value) (Value, error) {
}
case *Set: // symmetric difference
if y, ok := y.(*Set); ok {
- set := new(Set)
- for xe := x.ht.head; xe != nil; xe = xe.next {
- if found, _ := y.Has(xe.key); !found {
- set.Insert(xe.key)
- }
- }
- for ye := y.ht.head; ye != nil; ye = ye.next {
- if found, _ := x.Has(ye.key); !found {
- set.Insert(ye.key)
- }
- }
- return set, nil
+ iter := y.Iterate()
+ defer iter.Done()
+ return x.SymmetricDifference(iter)
}
}
@@ -1640,9 +1659,14 @@ func interpolate(format string, x Value) (Value, error) {
index++
}
- if index < nargs {
+ if index < nargs && !is[Mapping](x) {
return nil, fmt.Errorf("too many arguments for format string")
}
return String(buf.String()), nil
}
+
+func is[T any](x any) bool {
+ _, ok := x.(T)
+ return ok
+}
diff --git a/starlark/eval_test.go b/starlark/eval_test.go
index 9ffd179..6678671 100644
--- a/starlark/eval_test.go
+++ b/starlark/eval_test.go
@@ -6,6 +6,7 @@ package starlark_test
import (
"bytes"
+ "errors"
"fmt"
"math"
"os/exec"
@@ -20,7 +21,6 @@ import (
starlarkmath "go.starlark.net/lib/math"
"go.starlark.net/lib/proto"
"go.starlark.net/lib/time"
- "go.starlark.net/resolve"
"go.starlark.net/starlark"
"go.starlark.net/starlarkstruct"
"go.starlark.net/starlarktest"
@@ -31,22 +31,21 @@ import (
)
// A test may enable non-standard options by containing (e.g.) "option:recursion".
-func setOptions(src string) {
- resolve.AllowGlobalReassign = option(src, "globalreassign")
- resolve.LoadBindsGlobally = option(src, "loadbindsglobally")
- resolve.AllowRecursion = option(src, "recursion")
- resolve.AllowSet = option(src, "set")
+func getOptions(src string) *syntax.FileOptions {
+ return &syntax.FileOptions{
+ Set: option(src, "set"),
+ While: option(src, "while"),
+ TopLevelControl: option(src, "toplevelcontrol"),
+ GlobalReassign: option(src, "globalreassign"),
+ LoadBindsGlobally: option(src, "loadbindsglobally"),
+ Recursion: option(src, "recursion"),
+ }
}
func option(chunk, name string) bool {
return strings.Contains(chunk, "option:"+name)
}
-// Wrapper is the type of errors with an Unwrap method; see https://golang.org/pkg/errors.
-type Wrapper interface {
- Unwrap() error
-}
-
func TestEvalExpr(t *testing.T) {
// This is mostly redundant with the new *.star tests.
// TODO(adonovan): move checks into *.star files and
@@ -113,7 +112,6 @@ func TestEvalExpr(t *testing.T) {
}
func TestExecFile(t *testing.T) {
- defer setOptions("")
testdata := starlarktest.DataFile("starlark", ".")
thread := &starlark.Thread{Load: load}
starlarktest.SetReporter(thread, t)
@@ -139,6 +137,7 @@ func TestExecFile(t *testing.T) {
"testdata/tuple.star",
"testdata/recursion.star",
"testdata/module.star",
+ "testdata/while.star",
} {
filename := filepath.Join(testdata, file)
for _, chunk := range chunkedfile.Read(filename, t) {
@@ -148,9 +147,8 @@ func TestExecFile(t *testing.T) {
"struct": starlark.NewBuiltin("struct", starlarkstruct.Make),
}
- setOptions(chunk.Source)
-
- _, err := starlark.ExecFile(thread, filename, chunk.Source, predeclared)
+ opts := getOptions(chunk.Source)
+ _, err := starlark.ExecFileOptions(opts, thread, filename, chunk.Source, predeclared)
switch err := err.(type) {
case *starlark.EvalError:
found := false
@@ -607,12 +605,10 @@ Error: cannot load crash.star: floored division by zero`
result = evalErr
}
- // TODO: use errors.Unwrap when go >=1.13 is everywhere.
- wrapper, isWrapper := err.(Wrapper)
- if !isWrapper {
+ err = errors.Unwrap(err)
+ if err == nil {
break
}
- err = wrapper.Unwrap()
}
return result
}
diff --git a/starlark/hashtable.go b/starlark/hashtable.go
index 252d21d..40f72bb 100644
--- a/starlark/hashtable.go
+++ b/starlark/hashtable.go
@@ -6,6 +6,7 @@ package starlark
import (
"fmt"
+ "math/big"
_ "unsafe" // for go:linkname hack
)
@@ -200,6 +201,57 @@ func (ht *hashtable) lookup(k Value) (v Value, found bool, err error) {
return None, false, nil // not found
}
+// count returns the number of distinct elements of iter that are elements of ht.
+func (ht *hashtable) count(iter Iterator) (int, error) {
+ if ht.table == nil {
+ return 0, nil // empty
+ }
+
+ var k Value
+ count := 0
+
+ // Use a bitset per table entry to record seen elements of ht.
+ // Elements are identified by their bucket number and index within the bucket.
+ // Each bitset gets one word initially, but may grow.
+ storage := make([]big.Word, len(ht.table))
+ bitsets := make([]big.Int, len(ht.table))
+ for i := range bitsets {
+ bitsets[i].SetBits(storage[i : i+1 : i+1])
+ }
+ for iter.Next(&k) && count != int(ht.len) {
+ h, err := k.Hash()
+ if err != nil {
+ return 0, err // unhashable
+ }
+ if h == 0 {
+ h = 1 // zero is reserved
+ }
+
+ // Inspect each bucket in the bucket list.
+ bucketId := h & (uint32(len(ht.table) - 1))
+ i := 0
+ for p := &ht.table[bucketId]; p != nil; p = p.next {
+ for j := range p.entries {
+ e := &p.entries[j]
+ if e.hash == h {
+ if eq, err := Equal(k, e.key); err != nil {
+ return 0, err
+ } else if eq {
+ bitIndex := i<<3 + j
+ if bitsets[bucketId].Bit(bitIndex) == 0 {
+ bitsets[bucketId].SetBit(&bitsets[bucketId], bitIndex, 1)
+ count++
+ }
+ }
+ }
+ }
+ i++
+ }
+ }
+
+ return count, nil
+}
+
// Items returns all the items in the map (as key/value pairs) in insertion order.
func (ht *hashtable) items() []Tuple {
items := make([]Tuple, 0, ht.len)
diff --git a/starlark/hashtable_test.go b/starlark/hashtable_test.go
index 3649f14..bcbc8e8 100644
--- a/starlark/hashtable_test.go
+++ b/starlark/hashtable_test.go
@@ -123,3 +123,17 @@ func testHashtable(tb testing.TB, sane map[int]bool) {
}
}
}
+
+func TestHashtableCount(t *testing.T) {
+ const count = 1000
+ ht := new(hashtable)
+ for i := 0; i < count; i++ {
+ ht.insert(MakeInt(i), None)
+ }
+
+ if c, err := ht.count(rangeValue{0, count, 1, count}.Iterate()); err != nil {
+ t.Error(err)
+ } else if c != count {
+ t.Errorf("count doesn't match: expected %d got %d", count, c)
+ }
+}
diff --git a/starlark/int.go b/starlark/int.go
index a264e9d..8f2b279 100644
--- a/starlark/int.go
+++ b/starlark/int.go
@@ -191,15 +191,16 @@ func (i Int) Hash() (uint32, error) {
return 12582917 * uint32(lo+3), nil
}
-// Required by the TotallyOrdered interface
-func (x Int) Cmp(v Value, depth int) (int, error) {
- y := v.(Int)
- xSmall, xBig := x.get()
- ySmall, yBig := y.get()
- if xBig != nil || yBig != nil {
- return x.bigInt().Cmp(y.bigInt()), nil
+// Cmp implements comparison of two Int values.
+// Required by the TotallyOrdered interface.
+func (i Int) Cmp(v Value, depth int) (int, error) {
+ j := v.(Int)
+ iSmall, iBig := i.get()
+ jSmall, jBig := j.get()
+ if iBig != nil || jBig != nil {
+ return i.bigInt().Cmp(j.bigInt()), nil
}
- return signum64(xSmall - ySmall), nil // safe: int32 operands
+ return signum64(iSmall - jSmall), nil // safe: int32 operands
}
// Float returns the float value nearest i.
@@ -211,6 +212,12 @@ func (i Int) Float() Float {
return Float(iBig.Uint64())
} else if iBig.IsInt64() {
return Float(iBig.Int64())
+ } else {
+ // Fast path for very big ints.
+ const maxFiniteLen = 1023 + 1 // max exponent value + implicit mantissa bit
+ if iBig.BitLen() > maxFiniteLen {
+ return Float(math.Inf(iBig.Sign()))
+ }
}
f, _ := new(big.Float).SetInt(iBig).Float64()
diff --git a/starlark/int_generic.go b/starlark/int_generic.go
index d54bc2a..244a60c 100644
--- a/starlark/int_generic.go
+++ b/starlark/int_generic.go
@@ -1,5 +1,4 @@
//go:build (!linux && !darwin && !dragonfly && !freebsd && !netbsd && !solaris) || (!amd64 && !arm64 && !mips64x && !ppc64 && !ppc64le && !loong64 && !s390x)
-// +build !linux,!darwin,!dragonfly,!freebsd,!netbsd,!solaris !amd64,!arm64,!mips64x,!ppc64,!ppc64le,!loong64,!s390x
package starlark
diff --git a/starlark/int_posix64.go b/starlark/int_posix64.go
index 36c56e5..70c949e 100644
--- a/starlark/int_posix64.go
+++ b/starlark/int_posix64.go
@@ -1,6 +1,4 @@
//go:build (linux || darwin || dragonfly || freebsd || netbsd || solaris) && (amd64 || arm64 || mips64x || ppc64 || ppc64le || loong64 || s390x)
-// +build linux darwin dragonfly freebsd netbsd solaris
-// +build amd64 arm64 mips64x ppc64 ppc64le loong64 s390x
package starlark
diff --git a/starlark/interp.go b/starlark/interp.go
index b41905a..d29e525 100644
--- a/starlark/interp.go
+++ b/starlark/interp.go
@@ -10,7 +10,6 @@ import (
"go.starlark.net/internal/compile"
"go.starlark.net/internal/spell"
- "go.starlark.net/resolve"
"go.starlark.net/syntax"
)
@@ -24,19 +23,19 @@ func (fn *Function) CallInternal(thread *Thread, args Tuple, kwargs []Tuple) (Va
// Postcondition: args is not mutated. This is stricter than required by Callable,
// but allows CALL to avoid a copy.
- if !resolve.AllowRecursion {
+ f := fn.funcode
+ if !f.Prog.Recursion {
// detect recursion
for _, fr := range thread.stack[:len(thread.stack)-1] {
// We look for the same function code,
// not function value, otherwise the user could
// defeat the check by writing the Y combinator.
- if frfn, ok := fr.Callable().(*Function); ok && frfn.funcode == fn.funcode {
+ if frfn, ok := fr.Callable().(*Function); ok && frfn.funcode == f {
return nil, fmt.Errorf("function %s called recursively", fn.Name())
}
}
}
- f := fn.funcode
fr := thread.frameAt(0)
// Allocate space for stack and locals.
diff --git a/starlark/library.go b/starlark/library.go
index 1c801be..4e73a40 100644
--- a/starlark/library.go
+++ b/starlark/library.go
@@ -40,7 +40,7 @@ func init() {
"True": True,
"False": False,
"abs": NewBuiltin("abs", abs),
- "any": NewBuiltin("any", any),
+ "any": NewBuiltin("any", any_),
"all": NewBuiltin("all", all),
"bool": NewBuiltin("bool", bool_),
"bytes": NewBuiltin("bytes", bytes_),
@@ -140,7 +140,17 @@ var (
}
setMethods = map[string]*Builtin{
- "union": NewBuiltin("union", set_union),
+ "add": NewBuiltin("add", set_add),
+ "clear": NewBuiltin("clear", set_clear),
+ "difference": NewBuiltin("difference", set_difference),
+ "discard": NewBuiltin("discard", set_discard),
+ "intersection": NewBuiltin("intersection", set_intersection),
+ "issubset": NewBuiltin("issubset", set_issubset),
+ "issuperset": NewBuiltin("issuperset", set_issuperset),
+ "pop": NewBuiltin("pop", set_pop),
+ "remove": NewBuiltin("remove", set_remove),
+ "symmetric_difference": NewBuiltin("symmetric_difference", set_symmetric_difference),
+ "union": NewBuiltin("union", set_union),
}
)
@@ -200,7 +210,7 @@ func all(thread *Thread, _ *Builtin, args Tuple, kwargs []Tuple) (Value, error)
}
// https://github.com/google/starlark-go/blob/master/doc/spec.md#any
-func any(thread *Thread, _ *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
+func any_(thread *Thread, _ *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
var iterable Iterable
if err := UnpackPositionalArgs("any", args, kwargs, 1, &iterable); err != nil {
return nil, err
@@ -2168,6 +2178,162 @@ func string_splitlines(_ *Thread, b *Builtin, args Tuple, kwargs []Tuple) (Value
return NewList(list), nil
}
+// https://github.com/google/starlark-go/blob/master/doc/spec.md#set·add.
+func set_add(_ *Thread, b *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
+ var elem Value
+ if err := UnpackPositionalArgs(b.Name(), args, kwargs, 1, &elem); err != nil {
+ return nil, err
+ }
+ if found, err := b.Receiver().(*Set).Has(elem); err != nil {
+ return nil, nameErr(b, err)
+ } else if found {
+ return None, nil
+ }
+ err := b.Receiver().(*Set).Insert(elem)
+ if err != nil {
+ return nil, nameErr(b, err)
+ }
+ return None, nil
+}
+
+// https://github.com/google/starlark-go/blob/master/doc/spec.md#set·clear.
+func set_clear(_ *Thread, b *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
+ if err := UnpackPositionalArgs(b.Name(), args, kwargs, 0); err != nil {
+ return nil, err
+ }
+ if b.Receiver().(*Set).Len() > 0 {
+ if err := b.Receiver().(*Set).Clear(); err != nil {
+ return nil, nameErr(b, err)
+ }
+ }
+ return None, nil
+}
+
+// https://github.com/google/starlark-go/blob/master/doc/spec.md#set·difference.
+func set_difference(_ *Thread, b *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
+ // TODO: support multiple others: s.difference(*others)
+ var other Iterable
+ if err := UnpackPositionalArgs(b.Name(), args, kwargs, 0, &other); err != nil {
+ return nil, err
+ }
+ iter := other.Iterate()
+ defer iter.Done()
+ diff, err := b.Receiver().(*Set).Difference(iter)
+ if err != nil {
+ return nil, nameErr(b, err)
+ }
+ return diff, nil
+}
+
+// https://github.com/google/starlark-go/blob/master/doc/spec.md#set_intersection.
+func set_intersection(_ *Thread, b *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
+ // TODO: support multiple others: s.difference(*others)
+ var other Iterable
+ if err := UnpackPositionalArgs(b.Name(), args, kwargs, 0, &other); err != nil {
+ return nil, err
+ }
+ iter := other.Iterate()
+ defer iter.Done()
+ diff, err := b.Receiver().(*Set).Intersection(iter)
+ if err != nil {
+ return nil, nameErr(b, err)
+ }
+ return diff, nil
+}
+
+// https://github.com/google/starlark-go/blob/master/doc/spec.md#set_issubset.
+func set_issubset(_ *Thread, b *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
+ var other Iterable
+ if err := UnpackPositionalArgs(b.Name(), args, kwargs, 0, &other); err != nil {
+ return nil, err
+ }
+ iter := other.Iterate()
+ defer iter.Done()
+ diff, err := b.Receiver().(*Set).IsSubset(iter)
+ if err != nil {
+ return nil, nameErr(b, err)
+ }
+ return Bool(diff), nil
+}
+
+// https://github.com/google/starlark-go/blob/master/doc/spec.md#set_issuperset.
+func set_issuperset(_ *Thread, b *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
+ var other Iterable
+ if err := UnpackPositionalArgs(b.Name(), args, kwargs, 0, &other); err != nil {
+ return nil, err
+ }
+ iter := other.Iterate()
+ defer iter.Done()
+ diff, err := b.Receiver().(*Set).IsSuperset(iter)
+ if err != nil {
+ return nil, nameErr(b, err)
+ }
+ return Bool(diff), nil
+}
+
+// https://github.com/google/starlark-go/blob/master/doc/spec.md#set·discard.
+func set_discard(_ *Thread, b *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
+ var k Value
+ if err := UnpackPositionalArgs(b.Name(), args, kwargs, 1, &k); err != nil {
+ return nil, err
+ }
+ if found, err := b.Receiver().(*Set).Has(k); err != nil {
+ return nil, nameErr(b, err)
+ } else if !found {
+ return None, nil
+ }
+ if _, err := b.Receiver().(*Set).Delete(k); err != nil {
+ return nil, nameErr(b, err) // set is frozen
+ }
+ return None, nil
+}
+
+// https://github.com/google/starlark-go/blob/master/doc/spec.md#set·pop.
+func set_pop(_ *Thread, b *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
+ if err := UnpackPositionalArgs(b.Name(), args, kwargs, 0); err != nil {
+ return nil, err
+ }
+ recv := b.Receiver().(*Set)
+ k, ok := recv.ht.first()
+ if !ok {
+ return nil, nameErr(b, "empty set")
+ }
+ _, err := recv.Delete(k)
+ if err != nil {
+ return nil, nameErr(b, err) // set is frozen
+ }
+ return k, nil
+}
+
+// https://github.com/google/starlark-go/blob/master/doc/spec.md#set·remove.
+func set_remove(_ *Thread, b *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
+ var k Value
+ if err := UnpackPositionalArgs(b.Name(), args, kwargs, 1, &k); err != nil {
+ return nil, err
+ }
+ if found, err := b.Receiver().(*Set).Delete(k); err != nil {
+ return nil, nameErr(b, err) // dict is frozen or key is unhashable
+ } else if found {
+ return None, nil
+ }
+ return nil, nameErr(b, "missing key")
+}
+
+// https://github.com/google/starlark-go/blob/master/doc/spec.md#set·symmetric_difference.
+func set_symmetric_difference(_ *Thread, b *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
+ var other Iterable
+ if err := UnpackPositionalArgs(b.Name(), args, kwargs, 0, &other); err != nil {
+ return nil, err
+ }
+ iter := other.Iterate()
+ defer iter.Done()
+ diff, err := b.Receiver().(*Set).SymmetricDifference(iter)
+ if err != nil {
+ return nil, nameErr(b, err)
+ }
+ return diff, nil
+}
+
// https://github.com/google/starlark-go/blob/master/doc/spec.md#set·union.
func set_union(_ *Thread, b *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
var iterable Iterable
diff --git a/starlark/profile.go b/starlark/profile.go
index 38da2b2..590a4e2 100644
--- a/starlark/profile.go
+++ b/starlark/profile.go
@@ -101,11 +101,11 @@ func StartProfile(w io.Writer) error {
return nil
}
-// StopProfiler stops the profiler started by a prior call to
+// StopProfile stops the profiler started by a prior call to
// StartProfile and finalizes the profile. It returns an error if the
// profile could not be completed.
//
-// StopProfiler must not be called concurrently with Starlark execution.
+// StopProfile must not be called concurrently with Starlark execution.
func StopProfile() error {
// Terminate the profiler goroutine and get its result.
close(profiler.events)
diff --git a/starlark/profile_test.go b/starlark/profile_test.go
index 515d7d4..773a384 100644
--- a/starlark/profile_test.go
+++ b/starlark/profile_test.go
@@ -7,7 +7,6 @@ package starlark_test
import (
"bytes"
"fmt"
- "io/ioutil"
"os"
"os/exec"
"strings"
@@ -19,12 +18,11 @@ import (
// TestProfile is a simple integration test that the profiler
// emits minimally plausible pprof-compatible output.
func TestProfile(t *testing.T) {
- prof, err := ioutil.TempFile("", "profile_test")
+ prof, err := os.CreateTemp(t.TempDir(), "profile_test")
if err != nil {
t.Fatal(err)
}
defer prof.Close()
- defer os.Remove(prof.Name())
if err := starlark.StartProfile(prof); err != nil {
t.Fatal(err)
}
diff --git a/starlark/testdata/benchmark.star b/starlark/testdata/benchmark.star
index 5d9af10..c448889 100644
--- a/starlark/testdata/benchmark.star
+++ b/starlark/testdata/benchmark.star
@@ -126,3 +126,42 @@ def bench_to_json_deep_list(b):
"Benchmark json.encode builtin with a list of deep input"
for _ in range(b.n):
json.encode(deep)
+
+def bench_issubset_unique_large_small(b):
+ "Benchmark set.issubset builtin"
+ s = set(range(10000))
+ for _ in range(b.n):
+ s.issubset(range(1000))
+
+def bench_issubset_unique_small_large(b):
+ "Benchmark set.issubset builtin"
+ s = set(range(1000))
+ for _ in range(b.n):
+ s.issubset(range(10000))
+
+def bench_issubset_unique_same(b):
+ "Benchmark set.issubset builtin"
+ s = set(range(1000))
+ for _ in range(b.n):
+ s.issubset(range(1000))
+
+def bench_issubset_duplicate_large_small(b):
+ "Benchmark set.issubset builtin"
+ s = set(range(10000))
+ l = list(range(200)) * 5
+ for _ in range(b.n):
+ s.issubset(range(1000))
+
+def bench_issubset_duplicate_small_large(b):
+ "Benchmark set.issubset builtin"
+ s = set(range(1000))
+ l = list(range(2000)) * 5
+ for _ in range(b.n):
+ s.issubset(l)
+
+def bench_issubset_duplicate_same(b):
+ "Benchmark set.issubset builtin"
+ s = set(range(1000))
+ l = list(range(200)) * 5
+ for _ in range(b.n):
+ s.issubset(l)
diff --git a/starlark/testdata/builtins.star b/starlark/testdata/builtins.star
index b55428e..c7188fa 100644
--- a/starlark/testdata/builtins.star
+++ b/starlark/testdata/builtins.star
@@ -196,7 +196,7 @@ assert.eq(getattr(hf, "x"), 2)
assert.eq(hf.x, 2)
# built-in types can have attributes (methods) too.
myset = set([])
-assert.eq(dir(myset), ["union"])
+assert.eq(dir(myset), ["add", "clear", "difference", "discard", "intersection", "issubset", "issuperset", "pop", "remove", "symmetric_difference", "union"])
assert.true(hasattr(myset, "union"))
assert.true(not hasattr(myset, "onion"))
assert.eq(str(getattr(myset, "union")), "<built-in method union of set value>")
diff --git a/starlark/testdata/int.star b/starlark/testdata/int.star
index 46c0ad0..f0e2cde 100644
--- a/starlark/testdata/int.star
+++ b/starlark/testdata/int.star
@@ -74,7 +74,6 @@ def compound():
x %= 3
assert.eq(x, 2)
- # use resolve.AllowBitwise to enable the ops:
x = 2
x &= 1
assert.eq(x, 0)
@@ -197,7 +196,6 @@ assert.fails(lambda: int("0x-4", 16), "invalid literal with base 16: 0x-4")
# bitwise union (int|int), intersection (int&int), XOR (int^int), unary not (~int),
# left shift (int<<int), and right shift (int>>int).
-# use resolve.AllowBitwise to enable the ops.
# TODO(adonovan): this is not yet in the Starlark spec,
# but there is consensus that it should be.
assert.eq(1 | 2, 3)
diff --git a/starlark/testdata/recursion.star b/starlark/testdata/recursion.star
index 3368614..7029ea0 100644
--- a/starlark/testdata/recursion.star
+++ b/starlark/testdata/recursion.star
@@ -6,38 +6,9 @@
load("assert.star", "assert")
-def sum(n):
- r = 0
- while n > 0:
- r += n
- n -= 1
- return r
-
def fib(n):
if n <= 1:
return 1
return fib(n-1) + fib(n-2)
-def while_break(n):
- r = 0
- while n > 0:
- if n == 5:
- break
- r += n
- n -= 1
- return r
-
-def while_continue(n):
- r = 0
- while n > 0:
- if n % 2 == 0:
- n -= 1
- continue
- r += n
- n -= 1
- return r
-
assert.eq(fib(5), 8)
-assert.eq(sum(5), 5+4+3+2+1)
-assert.eq(while_break(10), 40)
-assert.eq(while_continue(10), 25)
diff --git a/starlark/testdata/set.star b/starlark/testdata/set.star
index bca4144..303b447 100644
--- a/starlark/testdata/set.star
+++ b/starlark/testdata/set.star
@@ -1,5 +1,5 @@
# Tests of Starlark 'set'
-# option:set
+# option:set option:globalreassign
# Sets are not a standard part of Starlark, so the features
# tested in this file must be enabled in the application by setting
@@ -9,13 +9,11 @@
# TODO(adonovan): support set mutation:
# - del set[k]
-# - set.remove
# - set.update
-# - set.clear
# - set += iterable, perhaps?
# Test iterator invalidation.
-load("assert.star", "assert")
+load("assert.star", "assert", "freeze")
# literals
# Parser does not currently support {1, 2, 3}.
@@ -48,7 +46,7 @@ y = set([3, 4, 5])
# set + any is not defined
assert.fails(lambda : x + y, "unknown.*: set \\+ set")
-# set | set (use resolve.AllowBitwise to enable it)
+# set | set
assert.eq(list(set("a".elems()) | set("b".elems())), ["a", "b"])
assert.eq(list(set("ab".elems()) | set("bc".elems())), ["a", "b", "c"])
assert.fails(lambda : set() | [], "unknown binary op: set | list")
@@ -67,12 +65,16 @@ assert.eq(list(x.union([5, 1])), [1, 2, 3, 5])
assert.eq(list(x.union((6, 5, 4))), [1, 2, 3, 6, 5, 4])
assert.fails(lambda : x.union([1, 2, {}]), "unhashable type: dict")
-# intersection, set & set (use resolve.AllowBitwise to enable it)
+# intersection, set & set or set.intersection(iterable)
assert.eq(list(set("a".elems()) & set("b".elems())), [])
assert.eq(list(set("ab".elems()) & set("bc".elems())), ["b"])
+assert.eq(list(set("a".elems()).intersection("b".elems())), [])
+assert.eq(list(set("ab".elems()).intersection("bc".elems())), ["b"])
-# symmetric difference, set ^ set (use resolve.AllowBitwise to enable it)
+# symmetric difference, set ^ set or set.symmetric_difference(iterable)
assert.eq(set([1, 2, 3]) ^ set([4, 5, 3]), set([1, 2, 4, 5]))
+assert.eq(set([1,2,3,4]).symmetric_difference([3,4,5,6]), set([1,2,5,6]))
+assert.eq(set([1,2,3,4]).symmetric_difference(set([])), set([1,2,3,4]))
def test_set_augmented_assign():
x = set([1, 2, 3])
@@ -100,7 +102,6 @@ assert.eq(x, x)
assert.eq(y, y)
assert.true(x != y)
assert.eq(set([1, 2, 3]), set([3, 2, 1]))
-assert.fails(lambda : x < y, "set < set not implemented")
# iteration
assert.true(type([elem for elem in x]), "list")
@@ -116,3 +117,82 @@ assert.eq(iter(), [1, 2, 3])
# sets are not indexable
assert.fails(lambda : x[0], "unhandled.*operation")
+
+# adding and removing
+add_set = set([1,2,3])
+add_set.add(4)
+assert.true(4 in add_set)
+freeze(add_set) # no mutation of frozen set because key already present
+add_set.add(4)
+assert.fails(lambda: add_set.add(5), "add: cannot insert into frozen hash table")
+
+# remove
+remove_set = set([1,2,3])
+remove_set.remove(3)
+assert.true(3 not in remove_set)
+assert.fails(lambda: remove_set.remove(3), "remove: missing key")
+freeze(remove_set)
+assert.fails(lambda: remove_set.remove(3), "remove: cannot delete from frozen hash table")
+
+# discard
+discard_set = set([1,2,3])
+discard_set.discard(3)
+assert.true(3 not in discard_set)
+assert.eq(discard_set.discard(3), None)
+freeze(discard_set)
+assert.eq(discard_set.discard(3), None) # no mutation of frozen set because key doesn't exist
+assert.fails(lambda: discard_set.discard(1), "discard: cannot delete from frozen hash table")
+
+
+# pop
+pop_set = set([1,2,3])
+assert.eq(pop_set.pop(), 1)
+assert.eq(pop_set.pop(), 2)
+assert.eq(pop_set.pop(), 3)
+assert.fails(lambda: pop_set.pop(), "pop: empty set")
+pop_set.add(1)
+pop_set.add(2)
+freeze(pop_set)
+assert.fails(lambda: pop_set.pop(), "pop: cannot delete from frozen hash table")
+
+# clear
+clear_set = set([1,2,3])
+clear_set.clear()
+assert.eq(len(clear_set), 0)
+freeze(clear_set) # no mutation of frozen set because its already empty
+assert.eq(clear_set.clear(), None)
+
+other_clear_set = set([1,2,3])
+freeze(other_clear_set)
+assert.fails(lambda: other_clear_set.clear(), "clear: cannot clear frozen hash table")
+
+# difference: set - set or set.difference(iterable)
+assert.eq(set([1,2,3,4]).difference([1,2,3,4]), set([]))
+assert.eq(set([1,2,3,4]).difference([1,2]), set([3,4]))
+assert.eq(set([1,2,3,4]).difference([]), set([1,2,3,4]))
+assert.eq(set([1,2,3,4]).difference(set([1,2,3])), set([4]))
+
+assert.eq(set([1,2,3,4]) - set([1,2,3,4]), set())
+assert.eq(set([1,2,3,4]) - set([1,2]), set([3,4]))
+
+# issuperset: set >= set or set.issuperset(iterable)
+assert.true(set([1,2,3]).issuperset([1,2]))
+assert.true(not set([1,2,3]).issuperset(set([1,2,4])))
+assert.true(set([1,2,3]) >= set([1,2,3]))
+assert.true(set([1,2,3]) >= set([1,2]))
+assert.true(not set([1,2,3]) >= set([1,2,4]))
+
+# proper superset: set > set
+assert.true(set([1, 2, 3]) > set([1, 2]))
+assert.true(not set([1,2, 3]) > set([1, 2, 3]))
+
+# issubset: set <= set or set.issubset(iterable)
+assert.true(set([1,2]).issubset([1,2,3]))
+assert.true(not set([1,2,3]).issubset(set([1,2,4])))
+assert.true(set([1,2,3]) <= set([1,2,3]))
+assert.true(set([1,2]) <= set([1,2,3]))
+assert.true(not set([1,2,3]) <= set([1,2,4]))
+
+# proper subset: set < set
+assert.true(set([1,2]) < set([1,2,3]))
+assert.true(not set([1,2,3]) < set([1,2,3]))
diff --git a/starlark/testdata/string.star b/starlark/testdata/string.star
index 1a38d62..e324780 100644
--- a/starlark/testdata/string.star
+++ b/starlark/testdata/string.star
@@ -172,7 +172,9 @@ assert.eq(gothash, wanthash)
# TODO(adonovan): ordered comparisons
# string % tuple formatting
+assert.eq("A" % (), "A")
assert.eq("A %d %x Z" % (123, 456), "A 123 1c8 Z")
+assert.eq("A" % {'unused': 123}, "A")
assert.eq("A %(foo)d %(bar)s Z" % {"foo": 123, "bar": "hi"}, "A 123 hi Z")
assert.eq("%s %r" % ("hi", "hi"), 'hi "hi"') # TODO(adonovan): use ''-quotation
assert.eq("%%d %d" % 1, "%d 1")
diff --git a/starlark/testdata/while.star b/starlark/testdata/while.star
new file mode 100644
index 0000000..676a3b4
--- /dev/null
+++ b/starlark/testdata/while.star
@@ -0,0 +1,37 @@
+# Tests of Starlark while statement.
+
+# This is a "chunked" file: each "---" effectively starts a new file.
+
+# option:while
+
+load("assert.star", "assert")
+
+def sum(n):
+ r = 0
+ while n > 0:
+ r += n
+ n -= 1
+ return r
+
+def while_break(n):
+ r = 0
+ while n > 0:
+ if n == 5:
+ break
+ r += n
+ n -= 1
+ return r
+
+def while_continue(n):
+ r = 0
+ while n > 0:
+ if n % 2 == 0:
+ n -= 1
+ continue
+ r += n
+ n -= 1
+ return r
+
+assert.eq(sum(5), 5+4+3+2+1)
+assert.eq(while_break(10), 40)
+assert.eq(while_continue(10), 25)
diff --git a/starlark/value.go b/starlark/value.go
index da21795..22a37c8 100644
--- a/starlark/value.go
+++ b/starlark/value.go
@@ -465,9 +465,11 @@ func isFinite(f float64) bool {
return math.Abs(f) <= math.MaxFloat64
}
-func (x Float) Cmp(y_ Value, depth int) (int, error) {
- y := y_.(Float)
- return floatCmp(x, y), nil
+// Cmp implements comparison of two Float values.
+// Required by the TotallyOrdered interface.
+func (f Float) Cmp(v Value, depth int) (int, error) {
+ g := v.(Float)
+ return floatCmp(f, g), nil
}
// floatCmp performs a three-valued comparison on floats,
@@ -1132,6 +1134,34 @@ func (x *Set) CompareSameType(op syntax.Token, y_ Value, depth int) (bool, error
case syntax.NEQ:
ok, err := setsEqual(x, y, depth)
return !ok, err
+ case syntax.GE: // superset
+ if x.Len() < y.Len() {
+ return false, nil
+ }
+ iter := y.Iterate()
+ defer iter.Done()
+ return x.IsSuperset(iter)
+ case syntax.LE: // subset
+ if x.Len() > y.Len() {
+ return false, nil
+ }
+ iter := y.Iterate()
+ defer iter.Done()
+ return x.IsSubset(iter)
+ case syntax.GT: // proper superset
+ if x.Len() <= y.Len() {
+ return false, nil
+ }
+ iter := y.Iterate()
+ defer iter.Done()
+ return x.IsSuperset(iter)
+ case syntax.LT: // proper subset
+ if x.Len() >= y.Len() {
+ return false, nil
+ }
+ iter := y.Iterate()
+ defer iter.Done()
+ return x.IsSubset(iter)
default:
return false, fmt.Errorf("%s %s %s not implemented", x.Type(), op, y.Type())
}
@@ -1149,11 +1179,28 @@ func setsEqual(x, y *Set, depth int) (bool, error) {
return true, nil
}
-func (s *Set) Union(iter Iterator) (Value, error) {
+func setFromIterator(iter Iterator) (*Set, error) {
+ var x Value
+ set := new(Set)
+ for iter.Next(&x) {
+ err := set.Insert(x)
+ if err != nil {
+ return set, err
+ }
+ }
+ return set, nil
+}
+
+func (s *Set) clone() *Set {
set := new(Set)
for e := s.ht.head; e != nil; e = e.next {
set.Insert(e.key) // can't fail
}
+ return set
+}
+
+func (s *Set) Union(iter Iterator) (Value, error) {
+ set := s.clone()
var x Value
for iter.Next(&x) {
if err := set.Insert(x); err != nil {
@@ -1163,6 +1210,72 @@ func (s *Set) Union(iter Iterator) (Value, error) {
return set, nil
}
+func (s *Set) Difference(other Iterator) (Value, error) {
+ diff := s.clone()
+ var x Value
+ for other.Next(&x) {
+ if _, err := diff.Delete(x); err != nil {
+ return nil, err
+ }
+ }
+ return diff, nil
+}
+
+func (s *Set) IsSuperset(other Iterator) (bool, error) {
+ var x Value
+ for other.Next(&x) {
+ found, err := s.Has(x)
+ if err != nil {
+ return false, err
+ }
+ if !found {
+ return false, nil
+ }
+ }
+ return true, nil
+}
+
+func (s *Set) IsSubset(other Iterator) (bool, error) {
+ if count, err := s.ht.count(other); err != nil {
+ return false, err
+ } else {
+ return count == s.Len(), nil
+ }
+}
+
+func (s *Set) Intersection(other Iterator) (Value, error) {
+ intersect := new(Set)
+ var x Value
+ for other.Next(&x) {
+ found, err := s.Has(x)
+ if err != nil {
+ return nil, err
+ }
+ if found {
+ err = intersect.Insert(x)
+ if err != nil {
+ return nil, err
+ }
+ }
+ }
+ return intersect, nil
+}
+
+func (s *Set) SymmetricDifference(other Iterator) (Value, error) {
+ diff := s.clone()
+ var x Value
+ for other.Next(&x) {
+ found, err := diff.Delete(x)
+ if err != nil {
+ return nil, err
+ }
+ if !found {
+ diff.Insert(x)
+ }
+ }
+ return diff, nil
+}
+
// toString returns the string form of value v.
// It may be more efficient than v.String() for larger values.
func toString(v Value) string {
@@ -1451,7 +1564,7 @@ func Iterate(x Value) Iterator {
// Bytes is the type of a Starlark binary string.
//
// A Bytes encapsulates an immutable sequence of bytes.
-// It is comparable, indexable, and sliceable, but not direcly iterable;
+// It is comparable, indexable, and sliceable, but not directly iterable;
// use bytes.elems() for an iterable view.
//
// In this Go implementation, the elements of 'string' and 'bytes' are
diff --git a/syntax/options.go b/syntax/options.go
new file mode 100644
index 0000000..51b2638
--- /dev/null
+++ b/syntax/options.go
@@ -0,0 +1,63 @@
+// Copyright 2023 The Bazel Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package syntax
+
+import _ "unsafe" // for linkname
+
+// FileOptions specifies various per-file options that affect static
+// aspects of an individual file such as parsing, name resolution, and
+// code generation. (Options that affect global dynamics are typically
+// controlled through [starlark.Thread].)
+//
+// The zero value of FileOptions is the default behavior.
+//
+// Many functions in this package come in two versions: the legacy
+// standalone function (such as [Parse]) uses [LegacyFileOptions],
+// whereas the more recent method (such as [Options.Parse]) honors the
+// provided options. The second form is preferred. In other packages,
+// the modern version is a standalone function with a leading
+// FileOptions parameter and the name suffix "Options", such as
+// [starlark.ExecFileOptions].
+type FileOptions struct {
+ // resolver
+ Set bool // allow references to the 'set' built-in function
+ While bool // allow 'while' statements
+ TopLevelControl bool // allow if/for/while statements at top-level
+ GlobalReassign bool // allow reassignment to top-level names
+ LoadBindsGlobally bool // load creates global not file-local bindings (deprecated)
+
+ // compiler
+ Recursion bool // disable recursion check for functions in this file
+}
+
+// TODO(adonovan): provide a canonical flag parser for FileOptions.
+// (And use it in the testdata "options:" strings.)
+
+// LegacyFileOptions returns a new FileOptions containing the current
+// values of the resolver package's legacy global variables such as
+// [resolve.AllowRecursion], etc.
+// These variables may be associated with command-line flags.
+func LegacyFileOptions() *FileOptions {
+ return &FileOptions{
+ Set: resolverAllowSet,
+ While: resolverAllowGlobalReassign,
+ TopLevelControl: resolverAllowGlobalReassign,
+ GlobalReassign: resolverAllowGlobalReassign,
+ Recursion: resolverAllowRecursion,
+ LoadBindsGlobally: resolverLoadBindsGlobally,
+ }
+}
+
+// Access resolver (legacy) flags, if they are linked in; false otherwise.
+var (
+ //go:linkname resolverAllowSet go.starlark.net/resolve.AllowSet
+ resolverAllowSet bool
+ //go:linkname resolverAllowGlobalReassign go.starlark.net/resolve.AllowGlobalReassign
+ resolverAllowGlobalReassign bool
+ //go:linkname resolverAllowRecursion go.starlark.net/resolve.AllowRecursion
+ resolverAllowRecursion bool
+ //go:linkname resolverLoadBindsGlobally go.starlark.net/resolve.LoadBindsGlobally
+ resolverLoadBindsGlobally bool
+)
diff --git a/syntax/parse.go b/syntax/parse.go
index f4c8fff..1183a03 100644
--- a/syntax/parse.go
+++ b/syntax/parse.go
@@ -23,19 +23,25 @@ const (
RetainComments Mode = 1 << iota // retain comments in AST; see Node.Comments
)
+// Parse calls the Parse method of LegacyFileOptions().
+// Deprecated: relies on legacy global variables.
+func Parse(filename string, src interface{}, mode Mode) (f *File, err error) {
+ return LegacyFileOptions().Parse(filename, src, mode)
+}
+
// Parse parses the input data and returns the corresponding parse tree.
//
-// If src != nil, ParseFile parses the source from src and the filename
+// If src != nil, Parse parses the source from src and the filename
// is only used when recording position information.
// The type of the argument for the src parameter must be string,
// []byte, io.Reader, or FilePortion.
-// If src == nil, ParseFile parses the file specified by filename.
-func Parse(filename string, src interface{}, mode Mode) (f *File, err error) {
+// If src == nil, Parse parses the file specified by filename.
+func (opts *FileOptions) Parse(filename string, src interface{}, mode Mode) (f *File, err error) {
in, err := newScanner(filename, src, mode&RetainComments != 0)
if err != nil {
return nil, err
}
- p := parser{in: in}
+ p := parser{options: opts, in: in}
defer p.in.recover(&err)
p.nextToken() // read first lookahead token
@@ -47,6 +53,12 @@ func Parse(filename string, src interface{}, mode Mode) (f *File, err error) {
return f, nil
}
+// ParseCompoundStmt calls the ParseCompoundStmt method of LegacyFileOptions().
+// Deprecated: relies on legacy global variables.
+func ParseCompoundStmt(filename string, readline func() ([]byte, error)) (f *File, err error) {
+ return LegacyFileOptions().ParseCompoundStmt(filename, readline)
+}
+
// ParseCompoundStmt parses a single compound statement:
// a blank line, a def, for, while, or if statement, or a
// semicolon-separated list of simple statements followed
@@ -54,13 +66,13 @@ func Parse(filename string, src interface{}, mode Mode) (f *File, err error) {
// ParseCompoundStmt does not consume any following input.
// The parser calls the readline function each
// time it needs a new line of input.
-func ParseCompoundStmt(filename string, readline func() ([]byte, error)) (f *File, err error) {
+func (opts *FileOptions) ParseCompoundStmt(filename string, readline func() ([]byte, error)) (f *File, err error) {
in, err := newScanner(filename, readline, false)
if err != nil {
return nil, err
}
- p := parser{in: in}
+ p := parser{options: opts, in: in}
defer p.in.recover(&err)
p.nextToken() // read first lookahead token
@@ -79,18 +91,24 @@ func ParseCompoundStmt(filename string, readline func() ([]byte, error)) (f *Fil
}
}
- return &File{Path: filename, Stmts: stmts}, nil
+ return &File{Options: opts, Path: filename, Stmts: stmts}, nil
+}
+
+// ParseExpr calls the ParseExpr method of LegacyFileOptions().
+// Deprecated: relies on legacy global variables.
+func ParseExpr(filename string, src interface{}, mode Mode) (expr Expr, err error) {
+ return LegacyFileOptions().ParseExpr(filename, src, mode)
}
// ParseExpr parses a Starlark expression.
// A comma-separated list of expressions is parsed as a tuple.
// See Parse for explanation of parameters.
-func ParseExpr(filename string, src interface{}, mode Mode) (expr Expr, err error) {
+func (opts *FileOptions) ParseExpr(filename string, src interface{}, mode Mode) (expr Expr, err error) {
in, err := newScanner(filename, src, mode&RetainComments != 0)
if err != nil {
return nil, err
}
- p := parser{in: in}
+ p := parser{options: opts, in: in}
defer p.in.recover(&err)
p.nextToken() // read first lookahead token
@@ -112,9 +130,10 @@ func ParseExpr(filename string, src interface{}, mode Mode) (expr Expr, err erro
}
type parser struct {
- in *scanner
- tok Token
- tokval tokenValue
+ options *FileOptions
+ in *scanner
+ tok Token
+ tokval tokenValue
}
// nextToken advances the scanner and returns the position of the
@@ -139,7 +158,7 @@ func (p *parser) parseFile() *File {
}
stmts = p.parseStmt(stmts)
}
- return &File{Stmts: stmts}
+ return &File{Options: p.options, Stmts: stmts}
}
func (p *parser) parseStmt(stmts []Stmt) []Stmt {
@@ -158,15 +177,17 @@ func (p *parser) parseStmt(stmts []Stmt) []Stmt {
func (p *parser) parseDefStmt() Stmt {
defpos := p.nextToken() // consume DEF
id := p.parseIdent()
- p.consume(LPAREN)
+ lparen := p.consume(LPAREN)
params := p.parseParams()
- p.consume(RPAREN)
+ rparen := p.consume(RPAREN)
p.consume(COLON)
body := p.parseSuite()
return &DefStmt{
Def: defpos,
Name: id,
+ Lparen: lparen,
Params: params,
+ Rparen: rparen,
Body: body,
}
}
@@ -275,10 +296,11 @@ func (p *parser) parseSimpleStmt(stmts []Stmt, consumeNL bool) []Stmt {
}
// small_stmt = RETURN expr?
-// | PASS | BREAK | CONTINUE
-// | LOAD ...
-// | expr ('=' | '+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=') expr // assign
-// | expr
+//
+// | PASS | BREAK | CONTINUE
+// | LOAD ...
+// | expr ('=' | '+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' | '<<=' | '>>=') expr // assign
+// | expr
func (p *parser) parseSmallStmt() Stmt {
switch p.tok {
case RETURN:
@@ -415,21 +437,23 @@ func (p *parser) consume(t Token) Position {
}
// params = (param COMMA)* param COMMA?
-// |
+//
+// |
//
// param = IDENT
-// | IDENT EQ test
-// | STAR
-// | STAR IDENT
-// | STARSTAR IDENT
+//
+// | IDENT EQ test
+// | STAR
+// | STAR IDENT
+// | STARSTAR IDENT
//
// parseParams parses a parameter list. The resulting expressions are of the form:
//
-// *Ident x
-// *Binary{Op: EQ, X: *Ident, Y: Expr} x=y
-// *Unary{Op: STAR} *
-// *Unary{Op: STAR, X: *Ident} *args
-// *Unary{Op: STARSTAR, X: *Ident} **kwargs
+// *Ident x
+// *Binary{Op: EQ, X: *Ident, Y: Expr} x=y
+// *Unary{Op: STAR} *
+// *Unary{Op: STAR, X: *Ident} *args
+// *Unary{Op: STARSTAR, X: *Ident} **kwargs
func (p *parser) parseParams() []Expr {
var params []Expr
for p.tok != RPAREN && p.tok != COLON && p.tok != EOF {
@@ -651,9 +675,10 @@ func init() {
}
// primary_with_suffix = primary
-// | primary '.' IDENT
-// | primary slice_suffix
-// | primary call_suffix
+//
+// | primary '.' IDENT
+// | primary slice_suffix
+// | primary call_suffix
func (p *parser) parsePrimaryWithSuffix() Expr {
x := p.parsePrimary()
for {
@@ -770,12 +795,13 @@ func (p *parser) parseArgs() []Expr {
return args
}
-// primary = IDENT
-// | INT | FLOAT | STRING | BYTES
-// | '[' ... // list literal or comprehension
-// | '{' ... // dict literal or comprehension
-// | '(' ... // tuple or parenthesized expression
-// | ('-'|'+'|'~') primary_with_suffix
+// primary = IDENT
+//
+// | INT | FLOAT | STRING | BYTES
+// | '[' ... // list literal or comprehension
+// | '{' ... // dict literal or comprehension
+// | '(' ... // tuple or parenthesized expression
+// | ('-'|'+'|'~') primary_with_suffix
func (p *parser) parsePrimary() Expr {
switch p.tok {
case IDENT:
@@ -836,9 +862,10 @@ func (p *parser) parsePrimary() Expr {
}
// list = '[' ']'
-// | '[' expr ']'
-// | '[' expr expr_list ']'
-// | '[' expr (FOR loop_variables IN expr)+ ']'
+//
+// | '[' expr ']'
+// | '[' expr expr_list ']'
+// | '[' expr (FOR loop_variables IN expr)+ ']'
func (p *parser) parseList() Expr {
lbrack := p.nextToken()
if p.tok == RBRACK {
@@ -865,8 +892,9 @@ func (p *parser) parseList() Expr {
}
// dict = '{' '}'
-// | '{' dict_entry_list '}'
-// | '{' dict_entry FOR loop_variables IN expr '}'
+//
+// | '{' dict_entry_list '}'
+// | '{' dict_entry FOR loop_variables IN expr '}'
func (p *parser) parseDict() Expr {
lbrace := p.nextToken()
if p.tok == RBRACE {
@@ -904,8 +932,9 @@ func (p *parser) parseDictEntry() *DictEntry {
}
// comp_suffix = FOR loopvars IN expr comp_suffix
-// | IF expr comp_suffix
-// | ']' or ')' (end)
+//
+// | IF expr comp_suffix
+// | ']' or ')' (end)
//
// There can be multiple FOR/IF clauses; the first is always a FOR.
func (p *parser) parseComprehensionSuffix(lbrace Position, body Expr, endBrace Token) Expr {
diff --git a/syntax/parse_test.go b/syntax/parse_test.go
index fedbb3e..197e905 100644
--- a/syntax/parse_test.go
+++ b/syntax/parse_test.go
@@ -9,7 +9,7 @@ import (
"bytes"
"fmt"
"go/build"
- "io/ioutil"
+ "os"
"path/filepath"
"reflect"
"strings"
@@ -472,7 +472,7 @@ var dataFile = func(pkgdir, filename string) string {
func BenchmarkParse(b *testing.B) {
filename := dataFile("syntax", "testdata/scan.star")
b.StopTimer()
- data, err := ioutil.ReadFile(filename)
+ data, err := os.ReadFile(filename)
if err != nil {
b.Fatal(err)
}
diff --git a/syntax/scan.go b/syntax/scan.go
index b080202..9549977 100644
--- a/syntax/scan.go
+++ b/syntax/scan.go
@@ -9,7 +9,6 @@ package syntax
import (
"fmt"
"io"
- "io/ioutil"
"log"
"math/big"
"os"
@@ -287,7 +286,7 @@ func readSource(filename string, src interface{}) ([]byte, error) {
case []byte:
return src, nil
case io.Reader:
- data, err := ioutil.ReadAll(src)
+ data, err := io.ReadAll(src)
if err != nil {
err = &os.PathError{Op: "read", Path: filename, Err: err}
return nil, err
@@ -296,7 +295,7 @@ func readSource(filename string, src interface{}) ([]byte, error) {
case FilePortion:
return src.Content, nil
case nil:
- return ioutil.ReadFile(filename)
+ return os.ReadFile(filename)
default:
return nil, fmt.Errorf("invalid source: %T", src)
}
diff --git a/syntax/scan_test.go b/syntax/scan_test.go
index 599dbbc..cfed6c9 100644
--- a/syntax/scan_test.go
+++ b/syntax/scan_test.go
@@ -8,7 +8,7 @@ import (
"bytes"
"fmt"
"go/build"
- "io/ioutil"
+ "os"
"path/filepath"
"strings"
"testing"
@@ -292,7 +292,7 @@ var dataFile = func(pkgdir, filename string) string {
func BenchmarkScan(b *testing.B) {
filename := dataFile("syntax", "testdata/scan.star")
b.StopTimer()
- data, err := ioutil.ReadFile(filename)
+ data, err := os.ReadFile(filename)
if err != nil {
b.Fatal(err)
}
diff --git a/syntax/syntax.go b/syntax/syntax.go
index 3756637..5bfbcec 100644
--- a/syntax/syntax.go
+++ b/syntax/syntax.go
@@ -70,7 +70,8 @@ type File struct {
Path string
Stmts []Stmt
- Module interface{} // a *resolve.Module, set by resolver
+ Module interface{} // a *resolve.Module, set by resolver
+ Options *FileOptions
}
func (x *File) Span() (start, end Position) {
@@ -99,9 +100,10 @@ func (*LoadStmt) stmt() {}
func (*ReturnStmt) stmt() {}
// An AssignStmt represents an assignment:
+//
// x = 0
// x, y = y, x
-// x += 1
+// x += 1
type AssignStmt struct {
commentsRef
OpPos Position
@@ -121,7 +123,9 @@ type DefStmt struct {
commentsRef
Def Position
Name *Ident
+ Lparen Position
Params []Expr // param = ident | ident=expr | * | *ident | **ident
+ Rparen Position
Body []Stmt
Function interface{} // a *resolve.Function, set by resolver
diff --git a/syntax/testdata/scan.star b/syntax/testdata/scan.star
index eec85c9..14fa6da 100644
--- a/syntax/testdata/scan.star
+++ b/syntax/testdata/scan.star
@@ -113,7 +113,7 @@ def _emit_generate_params_action(cmds, ctx, fn):
]
cmds_all += cmds
cmds_all_str = "\n".join(cmds_all) + "\n"
- f = ctx.new_file(ctx.configuration.bin_dir, fn)
+ f = ctx.actions.declare_file(fn)
ctx.file_action(
output = f,
content = cmds_all_str,
@@ -254,7 +254,10 @@ def _emit_go_cover_action(ctx, sources):
continue
cover_var = "GoCover_%d" % count
- out = ctx.new_file(src, src.basename[:-3] + "_" + cover_var + ".cover.go")
+ out = ctx.actions.declare_file(
+ src.basename[:-3] + "_" + cover_var + ".cover.go",
+ sibling = src,
+ )
outputs += [out]
ctx.action(
inputs = [src] + ctx.files.toolchain,
@@ -306,13 +309,16 @@ def go_library_impl(ctx):
extra_objects = [cgo_object.cgo_obj] if cgo_object else []
for src in asm_srcs:
- obj = ctx.new_file(src, "%s.dir/%s.o" % (ctx.label.name, src.basename[:-2]))
+ obj = ctx.actions.declare_file(
+ "%s.dir/%s.o" % (ctx.label.name, src.basename[:-2]),
+ sibling = src,
+ )
_emit_go_asm_action(ctx, src, asm_hdrs, obj)
extra_objects += [obj]
lib_name = _go_importpath(ctx) + ".a"
- out_lib = ctx.new_file(lib_name)
- out_object = ctx.new_file(ctx.label.name + ".o")
+ out_lib = ctx.actions.declare_file(lib_name)
+ out_object = ctx.actions.declare_file(ctx.label.name + ".o")
search_path = out_lib.path[:-len(lib_name)]
gc_goopts = _gc_goopts(ctx)
transitive_go_libraries = depset([out_lib])
@@ -531,9 +537,9 @@ def go_test_impl(ctx):
test into a binary."""
lib_result = go_library_impl(ctx)
- main_go = ctx.new_file(ctx.label.name + "_main_test.go")
- main_object = ctx.new_file(ctx.label.name + "_main_test.o")
- main_lib = ctx.new_file(ctx.label.name + "_main_test.a")
+ main_go = ctx.actions.declare_file(ctx.label.name + "_main_test.go")
+ main_object = ctx.actions.declare_file(ctx.label.name + "_main_test.o")
+ main_lib = ctx.actions.declare_file(ctx.label.name + "_main_test.a")
go_import = _go_importpath(ctx)
cmds = [
@@ -754,7 +760,7 @@ def _cgo_filter_srcs_impl(ctx):
for src in srcs:
stem, _, ext = src.path.rpartition(".")
dst_basename = "%s.filtered.%s" % (stem, ext)
- dst = ctx.new_file(src, dst_basename)
+ dst = ctx.actions.declare_file(dst_basename, sibling = src)
cmds += [
"if '%s' -cgo -quiet '%s'; then" %
(ctx.executable._filter_tags.path, src.path),
@@ -824,7 +830,7 @@ def _cgo_codegen_impl(ctx):
p = _pkg_dir(ctx.label.workspace_root, ctx.label.package) + "/"
if p == "./":
p = "" # workaround when cgo_library in repository root
- out_dir = (ctx.configuration.genfiles_dir.path + "/" +
+ out_dir = (ctx.configuration.bin_dir.path + "/" +
p + ctx.attr.outdir)
cc = ctx.fragments.cpp.compiler_executable
cmds = [
@@ -905,7 +911,6 @@ _cgo_codegen_rule = rule(
),
},
fragments = ["cpp"],
- output_to_genfiles = True,
)
def _cgo_codegen(
diff --git a/syntax/walk_test.go b/syntax/walk_test.go
index 00d9784..ec62473 100644
--- a/syntax/walk_test.go
+++ b/syntax/walk_test.go
@@ -96,7 +96,7 @@ for o in [p for q, r in s if t]:
})
fmt.Println(strings.Join(idents, " "))
- // The identifer 'a' appears in both LoadStmt.From[0] and LoadStmt.To[0].
+ // The identifier 'a' appears in both LoadStmt.From[0] and LoadStmt.To[0].
// Output:
// a a b c d e f g h i j k l m n o p q r s t u v w x y z
More details
Historical runs
- failed: build failed stage apt-get-dist-upgrade
- quilt-patch-out-of-date: Quilt patch 01-Revert-lib-proto-a-Starlark-package-for-protobuf-proce.patch no longer applies
- quilt-refresh-error: An error occurred refreshing quilt patch 01-Revert-lib-proto-a-Starlark-package-for-protobuf-proce.patch: Applying patch debian/patches/01-Revert-lib-proto-a-Starlark-package-for-protobuf-proce.patch patching file lib/proto/cmd/star2proto/star2proto.go Hunk #1 FAILED at 1. Not deleting file lib/proto/cmd/star2proto/star2proto.go as content differs from patch 1 out of 1 hunk FAILED -- rejects in file lib/proto/cmd/star2proto/star2proto.go patching file lib/proto/proto.go Hunk #1 FAILED at 1. Not deleting file lib/proto/proto.go as content differs from patch 1 out of 1 hunk FAILED -- rejects in file lib/proto/proto.go Patch debian/patches/01-Revert-lib-proto-a-Starlark-package-for-protobuf-proce.patch does not apply (enforce with -f)