177 KiB
Expr
TODO: Work in progress (last update on 2024/06/21, 05:40 a.m.)
1. Expr
Expr is a GO package capable of analysing, interpreting and calculating expressions.
1.1. Concepts and terminology
Expressions are texts containing sequences of operations represented by a syntax very similar to that of most programming languages. Expr package provides these macro functions:
-
Scanner — Its input is a text. It scans expression text characters to produce a flow of logical symbol and related attributes, aka tokens.
-
Parser — Parser input is the token flow coming from the scanner. It analyses the token flow verifyng if it complies with the Expr syntax. If that is the case, the Parser generates the Abstract Syntax Tree (AST). This is tree data structure that represents the components of an expressions and how they are related one each other.
-
Calculator. Its input is the AST. It computes the parsed expression contained in the AST and returns the result or an error.
1.1.1. Variables
Expr supports variables. The result of an expression can be stored in a variable and reused in other espressions simply specifying the name of the variable as an operand.
1.1.2. Multi-expression
An input text valid for Expr can contain more than an expression. Expressions are separated by ; (semicolon). When an input contains two or more expressions it is called multi-expression.
Expr parses and computes each expression of a multi-espression, from the left to the right. If all expressions are computed without errors, it only returns the value of the last, the right most.
The result of each expression of a multi-expression is stored in an automatic variable named last. In this way, each expression can refer to the result of the previous one without the need to assign that value to a new dedicated variable.
1.1.3. Calculation context
All objects, such as variables and functions, created during the calculation of an expression are stored in a memory called context.
The expression context is analogous to the stack-frame of other programming languages. When a function is called, a new context is allocated to store local definitions.
Function contexts are created by cloning the calling context. More details on this topic are given later in this document.
Expr creates and keeps a inner global context where it stores imported functions, either from builtin or plugin modules. To perform calculations, the calling program must provide its own context; this is the main context. All calculations take place in this context. As mentioned eralier, when a function is called, a new context is created by cloning the calling context. The created context can be called function context.
Imported functions are registerd in the global context. When an expression first calls an imported function, that function is linked to the current context; this can be the main context or a function context.
1.2. dev-expr test tool
Before we begin to describe the syntax of Expr, it is worth introducing dev-expr because it will be used to show many examples of expressions.
dev-expr is a simple program that can be used to evaluate expressions interactively. As its name suggests, it was created for testing purpose. In fact, in additon to the automatic verification test suite based on the Go test framework, dev-expr provided an important aid for quickly testing of new features during their development.
dev-expr can work as a REPL, Read-Execute-Print-Loop, or it can process expression acquired from files or standard input.
The program can be downloaded from dev-expr.
Here are some examples of execution.
dev-expr in REPL mode and ask for help# Type 'exit' or Ctrl+D to quit the program.
[user]$ ./dev-expr
dev-expr -- Expressions calculator v1.12.0(build 1),2024/09/14 (celestino.amoroso@portale-stac.it)
Based on the Expr package v0.26.0
Type help to get the list of available commands
See also https://git.portale-stac.it/go-pkg/expr/src/branch/main/README.adoc
>>> help
--- REPL commands:
base -- Set the integer output base: 2, 8, 10, or 16
exit -- Exit the program
help -- Show command list
ml -- Enable/Disable multi-line output
mods -- List builtin modules
output -- Enable/Disable printing expression results. Options 'on', 'off', 'status'
source -- Load a file as input
tty -- Enable/Disable ansi output (1)
--- Command line options:
-b <builtin> Import builtin modules.
<builtin> can be a list of module names or a glob-pattern.
Use the special value 'all' or the pattern '*' to import all modules.
-e <expression> Evaluate <expression> instead of standard-input
-i Force REPL operation when all -e occurences have been processed
-h, --help, help Show this help menu
-m, --modules List all builtin modules
--noout Disable printing of expression results
-p Print prefix form
-t Print tree form (2)
-v, --version Show program version
>>>| 1 | Only available for single fraction values |
| 2 | Work in progress |
[user]$ ./dev-expr
dev-expr -- Expressions calculator v1.10.0(build 14),2024/06/17 (celestino.amoroso@portale-stac.it)
Based on the Expr package v0.19.0
Type help to get the list of available commands
See also https://git.portale-stac.it/go-pkg/expr/src/branch/main/README.adoc
>>> 2+3
5
>>> 2+3*(4-1.5)
9.5
>>> 0xFD + 0b1 + 0o1 (1)
255
>>> 1|2 + 2|3 (2)
7|6
>>> ml (3)
>>> 1|2 + 2|3
7
-
6
>>> 4+2 but 5|2+0.5 (4)
3
>>> 4+2; 5|2+0.5 (5)
3
>>>| 1 | Number bases: 0x = hexadecimal, 0o = octal, 0b = binary. |
| 2 | Fractions: numerator | denominator. |
| 3 | Activate multi-line output of fractions. |
| 4 | But operator, see but operator. |
| 5 | Multi-expression: the same result of the previous single expression but this it is obtained with two separated calculations. |
2. Data types
Expr has its type system which is a subset of Golang’s type system. It supports numerical, string, relational, boolean expressions, and mixed-type lists and maps.
2.1. Numbers
Expr supports three type of numbers:
-
Integers
-
Floats
-
Fractions
In mixed operations involving integers, fractions and floats, automatic type promotion to the largest type is performed.
2.1.1. Integers
Expr's integers are a subset of the integer set. Internally they are stored as Golang int64 values.
integer = [sign] digit-seq
sign = "+" | "-"
digit-seq = dec-seq | bin-seq | oct-seq | hex-seq
dec-seq = {dec-digit}
dec-digit = "0"|"1"|…|"9"
bin-seq = "0b"{bin-digit}
bin-digit = "0"|"1"
oct-seq = "0o"{oct-digit}
oct-digit = "0"|"1"|…|"7"
hex-seq = "0x"{hex-digit}
hex-digit = "0"|"1"|…|"9"|"a"|…|"z"|"A"|…|"Z"
Value range: -9223372036854775808 to 9223372036854775807
Symbol |
Operation |
Description |
Examples |
|
Sum |
Add two values |
|
|
Subtraction |
Subtract the right value from the left one |
|
|
Product |
Multiply two values |
|
|
Integer division |
Divide the left value by the right one(*) |
|
|
Modulo |
Remainder of the integer division |
|
(*) See also the float division ./ below.
2.1.2. Floats
Expr's floats are a subset of the rational number set. Note that they can’t hold the exact value of an unlimited number; floats can only approximate them. Internally floats are stored as Golang’s float64 values.
float = [sign] dec-seq "." [dec-seq] [("e"|"E") [sign] dec-seq]
sign = "+" | "-"
dec-seq = see-integer-literal-syntax
>>> 1.0
1
>>> 0.123
0.123
>>> 4.5e+3
4500
>>> 4.5E-33
4.5e-33
>>> 4.5E-3
0.0045
>>> 4.5E10
4.5e+10
Symbol |
Operation |
Description |
Examples |
|
Sum |
Add two values |
|
|
Subtraction |
Subtract the right value from the left one |
|
|
Product |
Multiply two values |
|
|
Float division |
Divide the left value by the right one |
|
|
Forced float division |
Force float division |
|
2.1.3. Fractions
Expr also supports fractions. Fraction literals are made with two integers separated by a vertical bar |.
fraction = [sign] (num-den-spec | float-spec)
sign = "+" | "-"
num-den-spec = digit-seq "|" digit-seq
float-spec = dec-seq "." [dec-seq] "(" dec-seq ")"
dec-seq = see-integer-literal-syntax
digit-seq = see-integer-literal-syntax
>>> 1 | 2
1|2
>>> 4|6 // Fractions are always reduced to their lowest terms
2|3
>>> 1|2 + 2|3
7|6
>>> 1|2 * 2|3
1|3
>>> 1|2 / 1|3
3|2
>>> 1|2 ./ 1|3 // Force decimal division
1.5
>>> -1|2
-1|2
>>> 1|-2 // Invalid sign specification
Eval Error: [1:3] infix operator "|" requires two non-nil operands, got 1
>>> 1|(-2)
-1|2
Fractions can be used together with integers and floats in expressions.
>>> 1|2 + 5
11|2
>>> 4 - 1|2
7|2
>>> 1.0 + 1|2
1.5
2.2. Strings
Strings are character sequences enclosed between two double quote ".
>>> "I’m a string"
I’m a string
>>> "123abc?!"
123abc?!
>>> "123\nabc"
123
abc
>>> "123\tabc"
123 abc
Some arithmetic operators also apply to strings.
| Symbol | Operation | Description | Examples |
|---|---|---|---|
|
concatenation |
Join two strings or two stringable values |
|
|
repeat |
Make n copy of a string |
|
The charanters in a string can be accessed using the square [] operator.
item = string-expr "[" integer-expr "]"
sub-string = string-expr "[" integer-expr ":" integer-expr "]"
>>> s="abcd" // assign the string to variable s
"abcd"
>>> s[1] // char at position 1 (starting from 0)
"b"
>>> s[-1] // char at position -1, the rightmost one
"d"
>>> #s // number of chars
4
>>> #"abc" // number of chars
3
>>> s[1:3] // chars from position 1 to position 3 excluded
"bc"
2.3. Booleans
Boolean data type has two values only: true and false. Relational and boolean expressions result in boolean values.
| Symbol | Operation | Description | Examples |
|---|---|---|---|
|
Equal |
True if the left value is equal to the right one |
|
|
Not Equal |
True if the left value is NOT equal to the right one |
|
|
Less |
True if the left value is less than the right one |
|
|
Less or Equal |
True if the left value is less than or equal to the right one |
|
|
Greater |
True if the left value is greater than the right one |
|
|
Greater or Equal |
True if the left value is greater than or equal to the right one |
|
(*) See also the in operator in the list and dictionary sections.
| Symbol | Operation | Description | Examples |
|---|---|---|---|
|
Not |
True if the right value is false |
|
|
And |
True if both left and right values are true |
|
|
Or |
True if at least one of the left and right values integers is true |
|
|
Currently, boolean operations are evaluated using short cut evaluation. This means that, if the left expression of the
|
2.4. Lists
Expr supports list of mixed-type values, also specified by normal expressions. Internally, Expr's lists are Go arrays.
list = empty-list | non-empty-list
empty-list = "[]"
non-empty-list = "[" any-value {"," any-value} "]"
>>> [1,2,3] // List of integers
[1, 2, 3]
>>> ["one", "two", "three"] // List of strings
["one", "two", "three"]
>>> ["one", 2, false, 4.1] // List of mixed-types
["one", 2, false, 4.1]
>>> ["one"+1, 2.0*(9-2)] // List of expressions
["one1", 14]
>>> [ [1,"one"], [2,"two"]] // List of lists
[[1, "one"], [2, "two"]]
| Symbol | Operation | Description | Examples |
|---|---|---|---|
|
Join |
Joins two lists |
|
|
Difference |
Left list without elements in the right list |
|
|
Front insertion |
Insert an item in front |
|
|
Back insertion |
Insert an item at end |
|
|
Item at index |
Item at given position |
|
|
Item in list |
True if item is in list |
|
|
Size |
Number of items in a list |
|
Array’s items can be accessed using the index [] operator.
item = list-expr "[" integer-expr "]"
slice = string-expr "[" integer-expr ":" integer-expr "]"
>>> [1,2,3][1]
2
>>> index=2; ["a", "b", "c", "d"][index]
c
>>> ["a", "b", "c", "d"][2:]
["c", "d"]
>>> list=[1,2,3]; list[1]
2
>>> ["one","two","three"][1]
two
>>> list=["one","two","three"]; list[2-1]
two
>>> list[1]="six"; list
["one", "six", "three"]
>>> list[-1]
three
>>> list[10]
Eval Error: [1:9] index 10 out of bounds
>>> #list
3
>>> "first" >> list
["first", "one", "six", "three"]
>>> list << "last"
["first", "one", "six", "three", "last"]
>>> "six" in list
true
>>> "ten" in list
false
>>> [1,2,3] + ["one", "two", "three"]
[1, 2, 3, "one", "two", "three"]
>>> [1,2,3,4] - [2,4]
[1, 3]
2.5. Dictionaries
The dictionary, or dict, data-type represents sets of pairs key/value. It is also known as map or associative array.
Dictionary literals are sequences of pairs separated by comma , enclosed between brace brackets.
dict = empty-dict | non-empty-dict
empty-dict = "{}"
non-empty-dict = "{" key-scalar ":" any-value {"," key-scalar ":" any-value} "}"
item = dict-expr "[" key-expr "]"
| Symbol | Operation | Description | Examples |
|---|---|---|---|
|
Join |
Joins two dicts |
|
|
Dict item value |
Item value of given key |
|
|
Key in dict |
True if key is in dict |
|
|
Size |
Number of items in a dict |
|
>>> {1:"one", 2:"two"}
{1: "one", 2: "two"}
>>> {"one":1, "two": 2}
{"one": 1, "two": 2}
>>> {"sum":1+2+3, "prod":1*2*3}
{"sum": 6, "prod": 6}
>>> {"one":1, "two":2}["two"]
2
>>> d={"one":1, "two":2}; d["six"]=6; d
{"two": 2, "one": 1, "six": 6}
>>> #d
3
3. Variables
Expr, like most programming languages, supports variables. A variable is an identifier with an assigned value. Variables are stored in contexts.
variable = identifier "=" any-value
identifier = alpha {(alpha)|dec-digit|"_"}
alpha = "a"|"b"|…"z"|"A"|"B"|…"Z"
The assign operator = returns the value assigned to the variable.
|
>>> a=1
1
>>> a_b=1+2
3
>>> a_b
3
>>> x = 5.2 * (9-3) // The assigned value here has the typical approximation error of the float data-type
31.200000000000003
>>> x = 1; y = 2*x
2
>>> _a=2
Parse Error: [1:2] unexpected token "_"
>>> 1=2
Parse Error: assign operator ("=") must be preceded by a variable
4. Other operations
4.1. ; operator
The semicolon operator ; is an infixed pseudo-operator. It evaluates the left expression first and then the right expression. The value of the latter is the final result.
multi-expression = expression {";" expression }
An expression that contains ; is called a multi-expression and each component expression is called a sub-expression.
Technically ; is not treated as a real operator. It acts as a separator in lists of expressions.
|
; can be used to set some variables before the final calculation.
|
>>> a=1; b=2; c=3; a+b+c
6
The value of each sub-expression is stored in the automatic variable last.
>>> 2+3; b=last+10; last
15
4.2. but operator
but is an infixed operator. Its operands can be expressions of any type. It evaluates the left expression first, then the right expression. The value of the right expression is the final result.
>>> 5 but 2
2
>>> x=2*3 but x-1
5.
but behavior is very similar to ;. The only difference is that ; is not a true operator and can’t be used inside parenthesis ( and ).
4.3. Assignment operator =
The assignment operator = is used to define variables or to change their value in the evaluation context (see ExprContext).
The value on the left side of = must be a variable identifier or an expression that evalutes to a variable. The value on the right side can be any expression and it becomes the result of the assignment operation.
>>> a=15+1
16
>>> L=[1,2,3]; L[1]=5; L
[1, 5, 3]
4.4. Selector operator ? : ::
The selector operator is very similar to the switch/case/default statement available in many programming languages.
selector-operator = select-expression "?" selector-case { ":" selector-case } ["::" default-multi-expression]
selector-case = [match-list] case-value
match-list = "[" item {"," items} "]"
item = expression
case-multi-expression = "{" multi-expression "}"
multi-expression = expression { ";" expression }
default-multi-expression = multi-expression
In other words, the selector operator evaluates the select-expression on the left-hand side of the ? symbol; it then compares the result obtained with the values listed in the match-list's, from left to right. If the comparision finds a match with a value in a match-list, the associated case-multi-expression is evaluted, and its result will be the final result of the selection operation.
The match lists are optional. In that case, the position, from left to right, of the selector-case is used as match-list. Of course, that only works if the select-expression results in an integer.
The : symbol (colon) is the separator of the selector-cases. Note that if the value of the select-expression does not match any match-list, an error will be issued. Therefore, it is strongly recommended to provide a default (multi-)expression introduced by the :: symbol (double-colon). Also note that the default expression has no match-list.
>>> 1 ? {"a"} : {"b"}
b
>>> 10 ? {"a"} : {"b"} :: {"c"}
c
>>> 10 ? {"a"} :[true, 2+8] {"b"} :: {"c"}
b
>>> 10 ? {"a"} :[true, 2+8] {"b"} :: [10] {"c"}
Parse Error: [1:34] case list in default clause
>>> 10 ? {"a"} :[10] {x="b" but x} :: {"c"}
b
>>> 10 ? {"a"} :[10] {x="b"; x} :: {"c"}
b
>>> 10 ? {"a"} : {"b"}
Eval Error: [1:3] no case catches the value (10) of the selection expression
4.5. Variable default value ??, ?=, and ?!
The left operand of first two operators, ?? and ?=, must be a variable. The right operator can be any expression. They return the value of the variable if this is defined; otherwise they return the value of the right expression.
| If the left variable is defined, the right expression is not evaluated at all. |
The ?? operator do not change the status of the left variable.
The ?= assigns the calculated value of the right expression to the left variable.
The third one, ?!, is the alternate operator. If the variable on the left size is not defined, it returns nil. Otherwise it returns the result of the expressione on the right side.
| If the left variable is NOT defined, the right expression is not evaluated at all. |
>>> var ?? (1+2)
3
>>> var
Eval Error: undefined variable or function "var"
>>> var ?= (1+2)
3
>>> var
3
>>> x ?! 5
nil
>>> x=1; x ?! 5
5
>>> y ?! (c=5); c
Eval Error: undefined variable or function "c"
These operators have a high priority, in particular higher than the operator =.
|
5. Priorities of operators
The table below shows all supported operators by decreasing priorities.
| Priority | Operators | Position | Operation | Operands and results |
|---|---|---|---|---|
ITEM |
|
Postfix |
List item |
list |
|
Postfix |
Dict item |
dict |
|
INC |
|
Postfix |
Post increment |
integer-variable |
|
Postfix |
Next item |
iterator |
|
DEFAULT |
|
Infix |
Default value |
variable |
|
Infix |
Default/assign value |
variable |
|
|
Infix |
Alternate value |
variable |
|
FACT |
|
Postfix |
Factorial |
integer |
SIGN |
|
Prefix |
Change-sign |
( |
|
Prefix |
Lenght-of |
|
|
|
Prefix |
Size-of |
|
|
SELECT |
|
Multi-Infix |
Case-Selector |
any-expr |
|
Multi-Infix |
Index-Selector |
int-expr |
|
FRACT |
|
Infix |
Fraction |
integer |
PROD |
|
Infix |
Product |
number |
|
Infix |
String-repeat |
string |
|
|
Infix |
Division |
number |
|
|
Infix |
Float-division |
number |
|
|
Infix |
Integer-remainder |
integer |
|
SUM |
|
Infix |
Sum |
number |
|
Infix |
String-concat |
(string|number) |
|
|
Infix |
List-join |
list |
|
|
Infix |
Dict-join |
dict |
|
|
Infix |
Subtraction |
number |
|
|
Infix |
List-difference |
list |
|
RELATION |
|
Infix |
Less |
comparable |
|
Infix |
less-equal |
comparable |
|
|
Infix |
Greater |
comparable |
|
|
Infix |
Greater-equal |
comparable |
|
|
Infix |
Equal |
comparable |
|
|
Infix |
Not-equal |
comparable |
|
|
Infix |
Member-of-list |
any |
|
|
Infix |
Key-of-dict |
any |
|
NOT |
|
Prefix |
Not |
|
AND |
|
Infix |
And |
boolean |
|
Infix |
And |
boolean |
|
OR |
|
Infix |
Or |
boolean |
|
Infix |
Or |
boolean |
|
ASSIGN |
|
Infix |
Assignment |
identifier |
|
Infix |
Front-insert |
any |
|
|
Infix |
Back-insert |
list |
|
BUT |
|
Infix |
But |
any |
RANGE |
|
Infix |
Index-range |
integer |
6. Functions
Functions in Expr are very similar to functions available in many programming languages. Currently, Expr supports two types of function, expr-functions and go-functions.
-
expr-functions are defined using Expr's syntax. They can be passed as arguments to other functions and can be returned from functions. Moreover, they bind themselves to the defining context, thus becoming closures.
-
go-functions are regular Golang functions callable from Expr expressions. They are defined in Golang source files called modules and compiled within the Expr package. To make Golang functions available in Expr contextes, it is required to activate the builtin module or to load the plugin module in which they are defined.
6.1. Expr function definition
A function is identified and referenced by its name. It can have zero or more parameter. Expr functions also support optional parameters and passing paramters by name.
function-definition = identifier "=" "func(" [formal-param-list] ")" "{" multi-expression "}"
formal-param_list = required-param-list [ "," optional-param-list ]
required-param-list = identifier { "," identifier }
optional-param-list = optional-parm { "," optional-param }
optional-param = param-name "=" any-expr
param-name = identifier
>>> // A simple function: it takes two parameters and returns their "sum"(*)
>>> sum = func(a, b){ a + b }
sum(a, b):any{}
(*) Since the plus, *+*, operator is defined for multiple data-types, the sum() function can be used for any pair of that types.
>>> // A more complex example: recursive calculation of the n-th value of Fibonacci’s sequence
>>> fib = func(n){ n ? [0] {0}: [1] {1} :: {fib(n-1)+fib(n-2)} }
fib(n):any{}
>>> // Same function fib() but entered by splitting it over mulple text lines
>>> fib = func(n){ \
... n ? \
... [0] {0} : \
... [1] {1} :: \
... { \
... fib(n-1) + fib(n-2) \
... } \
... }
fib(n):any{}
>>> // Required and optional parameters
>>> measure = func(value, unit="meter"){ value + " " + unit + (value > 1) ? [true] {"s"} :: {""}}
measure(value, unit="meter"):any{}
6.2. Golang function definition
Description of how to define Golan functions and how to bind them to Expr are topics treated in another document that I’ll write, one day, maybe.
6.3. Function calls
To call a function, either Expr or Golang type, it is necessary to specify its name and, at least, its required parameters.
function-call = identifier "(" actual-param-list ")"
actual-param-list = [positional-params] [named-parameters]
positional-params = any-value { "," any-value }
named-params = param-name "=" any-value { "," param-name "=" any-value }
param-name = identifier
sum() functions defined above>>> // sum of two integers
>>> sum(-6, 2)
-4
>>> // same as above but passing the parameters by name
>>> sum(a=-6, b=2)
-4
>>> // again, but swapping parameter positions (see the diff() examples below)
>>> sum(b=2, a=-6)
-4
>>> // sum of a fraction and an integer
>>> sum(3|2, 2)
7|2
>>> // sum of two strings
>>> sum("bye", "-bye")
"bye-bye"
>>> // sum of two lists
>>> sum(["one", 1], ["two", 2])
["one", 1, "two", 2]
>>> // diff(a,b) calculates a-b
>>> diff = func(a,b){a-b}
diff(a, b):any{}
>>> // simple invocation
>>> diff(10,8)
2
>>> // swapped parameters passed by name
>>> diff(b=8,a=10)
2
fib() function defined above>>> // Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, …
>>> fib(6)
8
>>> fib(9)
34
measure() functions defined above>>> // simple call
>>> measure(10,"litre")
"10 litres"
>>> // accept the default unit
>>> measure(8)
"8 meters"
>>> // without the required parameter 'value'
>>> measure(unit="degrees"))
Eval Error: measure(): missing params — value
>>> factory = func(n=2){ func(x){x*n} }
factory(n=2):any{}
>>> double = factory()
double(x):any{}
>>> triple = factory(3)
triple(x):any{}
>>> double(5)
10
>>> triple(5)
15
Functions compute values in a local context (scope) that do not make effects on the calling context. This is the normal behavior. Using the reference operator @ it is possibile to export local definition to the calling context.
7. Iterators
TODO: function calls operations
8. Builtins
TODO: builtins
8.2. import()
import(<source-file>) loads the multi-expression contained in the specified source and returns its value.
9. Plugins
TODO: plugins