= Expr
Expressions calculator
:authors: Celestino Amoroso
:email: celestino.amoroso@gmail.com
:docinfo: shared
:encoding: utf-8
:toc: right
:toclevels: 4
//:toc-title: Indice Generale
:icons: font
:icon-set: fi
:numbered:
:data-uri:
//:table-caption: Tabella
//:figure-caption: Diagramma
:docinfo1:
:sectlinks:
:sectanchors:
:source-highlighter: rouge
// :rouge-style: gruvbox
:rouge-style: manni
:stylesdir: /home/share/s3-howto/styles
:stylesheet: adoc-colony.css

// Workaround to manage double-column in back-tick quotes
:2colon: ::
// Workaround to manage double-plus in back-tick quotes
:plusplus: ++
// Workaround to manage asterisk in back-tick quotes
:star: *

:sp: &#160;
:2sp: &#160;&#160;
:4sp: &#160;&#160;&#160;&#160;

:show_todo: 

toc::[]

#TODO: Work in progress#

== Expr
_Expr_ is a GO package that can analyze, interpret and calculate expressions.


=== Concepts and terminology

Expressions are texts containing sequences of operations represented by a syntax very similar to that of most programming languages. _Expr_ package provides these macro functions:

* *_Scanner_* -- Its input is a text. It scans expression text characters to produce a flow of logical symbol and related attributes, aka tokens.
* *_Parser_* -- Parser input is the token flow coming from the scanner. It analyses the token flow verifying if it complies with the _Expr_ syntax. If that is the case, the Parser generates the Abstract Syntax Tree (AST). This is tree data structure that represents the components of an expression and how they are related to each other.
* *_Calculator_*. Its input is the AST. It computes the parsed expression contained in the AST and returns the result or an error.

image::expression-diagram.png[]

==== Variables
_Expr_ supports variables. The result of an expression can be stored in a variable and reused in other expressions by simply specifying the name of the variable as an operand.

==== Multi-expression
An input text valid for _Expr_ can contain more than an expression. Expressions are separated by [blue]`;` (semicolon). When an input contains two or more expressions it is called _multi-expression_.

_Expr_ parses and computes each expression of a multi-expression, from the left to the right. If all expressions are computed without errors, it only returns the value of the last, the right most.

The result of each expression of a multi-expression is stored in an automatic variable named _last_. In this way, each expression can refer to the result of the previous one without the need to assign that value to a new dedicated variable.

==== Calculation context
All objects, such as variables and functions, created during the calculation of an expression are stored in a memory block called _context_.

The expression context is analogous to the stack-frame of other programming languages. When a function is called, a new context is allocated to store local definitions.

Function contexts are created by cloning the calling context. More details on this topic are given later in this document.

_Expr_ creates and keeps a inner _global context_ where it stores imported functions, either from builtin or plugin modules. To perform calculations, the user program must provide its own context; this is the _main context_. All calculations take place in this context. As mentioned earlier, when a function is called, a new context is created by cloning the calling context. The created context can be called _function context_.

Imported functions are registered in the _global context_. When an expression first calls an imported function, that function is linked to the current context; this can be the _main context_ or a _function context_.

===== Inspecting contexts
_Expr_ provides the operator [blue]_$$_ that returns the current context. This can be used to inspect the content of the context, for example to check the value of a variable or to see which functions are currently linked to the context. This operator is primarily intended for debugging purposes.

An interactive tool like `ecli` (see <<sec_ecli>>) can be used to inspect contexts interactively.


.Example: inspecting contexts
`>>>` [blue]`$$` +
[green]`{"variables": {"ls": "[10, 20, 30]", "it": "$([#3])", "last": "-1"}, "functions": {"about": "about():string{}", "bin": "bin(value, digits=8):integer{}", "ctrl": "ctrl(prop, value):any{}", "ctrlList": "ctrlList():list-of-strings{}", "envGet": "envGet(name):string{}", "envSet": "envSet(name, value):string{}"}}` +

`>>>` [gray]_// Let use the *ml* command to activate multi-line output of contexts, which is more readable._ +
`>>>` [blue]`ml` +
`>>>` [blue]`$$` +
[green]`{` +
  [green]`{2sp}"variables": {` +
  [green]`{4sp}"last": {"variables": {"last": "-1", "ls": "[10, 20, 30]", "it": "$([#3])"}, "functions": {"ctrlList": "ctrlList():list-of-strings{}", "envGet": "envGet(name):string{}", "envSet": "envSet(name, value):string{}", "about": "about():string{}", "bin": "bin(value, digits=8):integer{}", "ctrl": "ctrl(prop, value):any{}"}},` +
  [green]`{4sp}"ls": [10, 20, 30],` +
  [green]`{4sp}"it": $([#3])` +
  [green]`{2sp}},` +
  [green]`{2sp}"functions": {` +
  [green]`{4sp}"ctrlList": ctrlList():list-of-strings{},` +
  [green]`{4sp}"envGet": envGet(name):string{},` +
  [green]`{4sp}"envSet": envSet(name, value):string{},` +
  [green]`{4sp}"about": about():string{},` +
  [green]`{4sp}"bin": bin(value, digits=8):integer{},` +
  [green]`{4sp}"ctrl": ctrl(prop, value):any{}` +
  [green]`{2sp}}` +
[green]`}`

To inspect the global context perform the [blue]`$$ global` operation.

////
.Example: list all functions whose name starts with "str"
`>>>` [blue]`builtin "string`  [gray]__// most functions in the builtin module *string* have names starting with "str".__ +
[green]`1`

:dollar: $
:2dollars: $$

`>>>` [blue]`($(($$global).functions) filter strStartsWith($_, "str"))` +
[green]`[` +
[green]`{2sp}"strEndsWith",` +
[green]`{2sp}"strJoin",` +
[green]`{2sp}"strLower",` +
[green]`{2sp}"strSplit",` +
[green]`{2sp}"strStartsWith",` +
[green]`{2sp}"strSub",` +
[green]`{2sp}"strTrim",` +
[green]`{2sp}"strUpper",` +
[green]`{2sp}"string"` +
[green]`]`
////

[[sec_ecli]]
=== `ecli` Expression Calculator Interactive Tool
Before we begin to describe the syntax of _Expr_, it is worth introducing _ecli_, former _dev-expr_, because it will be used to show many examples of expressions.

`ecli` is a simple program that can be used to evaluate expressions interactively. As its original name _dev-expr_ suggests, it was created for testing purpose.  In fact, in addition to the automatic verification test suite based on the Go test framework, `ecli` provides an important aid for quickly testing of new features during their development.

`ecli` can work as a _REPL_, _**R**ead-**E**xecute-**P**rint-**L**oop_, or it can process expression acquired from files or standard input.

The program can be downloaded from https://git.portale-stac.it/go-pkg/-/packages/generic/ecli/[ecli].

Here are some examples of execution.

.Run `ecli` in REPL mode and ask for help
[source,shell]
----
# Type 'exit' or Ctrl+D to quit the program.

[user]$ ./ecli
ecli -- Expressions calculator v1.12.0(build 1),2024/09/14 (celestino.amoroso@portale-stac.it)
	Based on the Expr package v0.26.0
	Type help to get the list of available commands
	See also https://git.portale-stac.it/go-pkg/expr/src/branch/main/README.adoc
>>> help
--- REPL commands:
        base -- Set the integer output base: 2, 8, 10, or 16
        exit -- Exit the program
        help -- Show command list
          ml -- Enable/Disable multi-line output
        mods -- List builtin modules
      output -- Enable/Disable printing expression results. Options 'on', 'off', 'status'
      source -- Load a file as input
         tty -- Enable/Disable ansi output <1>

--- Command line options:
    -b <builtin>       Import builtin modules.
                       <builtin> can be a list of module names or a glob-pattern.
                       Use the special value 'all' or the pattern '*' to import all modules.
    -e <expression>    Evaluate <expression> instead of standard-input
    -i                 Force REPL operation when all -e occurences have been processed
    -h, --help, help   Show this help menu
    -m, --modules      List all builtin modules
    --noout            Disable printing of expression results
    -p                 Print prefix form
    -t                 Print tree form <2>
    -v, --version      Show program version
>>>
----

<1> Only available for single fraction values
<2> Work in progress

.REPL examples
[source,shell]
----
[user]$ ./ecli
ecli -- Expressions calculator v1.10.0(build 14),2024/06/17 (celestino.amoroso@portale-stac.it)
	Based on the Expr package v0.19.0
	Type help to get the list of available commands
	See also https://git.portale-stac.it/go-pkg/expr/src/branch/main/README.adoc

>>> 2+3
5
>>> 2+3*(4-1.5)
9.5
>>> 0xFD + 0b1 + 0o1 <1>
255
>>> 1:2 + 2:3 <2>
7:6
>>> ml <3>
>>> 1:2 + 2:3
7
-
6
>>> 4+2 but 5:2+0.5 <4>
3
>>> 4+2; 5:2+0.5 <5>
3
>>>
----

<1> Number bases: 0x = _hexadecimal_, 0o = _octal_, 0b = _binary_.
<2> Fractions: _numerator_ : _denominator_.
<3> Activate multi-line output of fractions.
<4> But operator, see <<_but_operator>>.
<5> Multi-expression: the same result as the previous single expression, but this time it is obtained with two separate calculations.

== Data types
_Expr_ has its type system which is approximately a subset of Golang's type system. It supports numerical, string, relational, boolean expressions, and mixed-type lists and maps.

=== Numbers
_Expr_ supports three type of numbers:

. [blue]#Integers#
. [blue]#Floats#
. [blue]#Fractions#

In mixed operations involving integers, fractions and floats, automatic type promotion to the larger type is performed.

==== Integers
__Expr__'s integers are a subset of the integer set. Internally they are stored as Golang _int64_ values.

.Integer literal syntax
====
*_integer_* = [_sign_] _digit-seq_ +
_sign_ = "**+**" | "**-**" +
_digit-seq_ = _dec-seq_ | _bin-seq_ | _oct-seq_ | _hex-seq_ +
_dec-seq_ = {__dec-digit__} +
_dec-digit_ = "**0**"|"**1**"|...|"**9**" +
_bin-seq_ = "**0b**"{__bin-digit__} +
_bin-digit_ = "**0**"|"**1**" +
_oct-seq_ = "**0o**"{__oct-digit__} +
_oct-digit_ = "**0**"|"**1**"|...|"**7**" +
_hex-seq_ = "**0x**"{__hex-digit__} +
_hex-digit_ = "**0**"|"**1**"|...|"**9**"|"**a**"|...|"**z**"|"**A**"|...|"**Z**"
====

Value range: *-9223372036854775808* to *9223372036854775807*

.Arithmetic operators
[cols="^1,^2,6,4"]
|===
| Symbol    | Operation     | Description               | Examples
| [blue]`+` | _Sum_         | Add two values<<note_int_plus_string,^*(1)*^>>            | [blue]`-1 + 2` -> _1_
| [blue]`-` | _Subtraction_ | Subtract the right value from the left one | [blue]`3 - 1` -> _2_
| [blue]`*` | _Product_     | Multiply two values<<note_string_repl,^*(2)*^>>       | [blue]`-1 * 2` -> _-2_
| [blue]`/` | _Integer division_    | Divide the left value by the right one<<note_float_division,^*(3)*^>> | [blue]`-11 / 2` -> _-5_
| [blue]`%` | _Modulo_      | Remainder of the integer division | [blue]`5 % 2` -> _1_
|===

====
[[note_int_plus_string]]
^*(1)*^ The sum operator [blue]`+` also supports adding an integer number to a string. In this case, the number is converted to a string and prependend or appended to the string, e.g. `"x" + 48` results in `"x48"`.

[[note_string_repl]]
^*(2)*^ The product operator also supports multiplying a string by an integer. In this case, the number represents how many times the string has to be repeated in the result, e.g. `"foo" * 3` returns `"foofoofoo"`.

[[note_float_division]]
^*(3)*^ See also the _float division_ [blue]`./` below.
====

==== Floats
__Expr__'s floats are a subset of the rational number set. Note that they can't hold the exact value of an unlimited number; floats can only approximate them. Internally floats are stored as Golang's _float64_ values.


.Float literal syntax
====
*_float_* = [_sign_] _dec-seq_ "**.**" [_dec-seq_] [("**e**"|"**E**") [_sign_] _dec-seq_] +
_sign_ = "**+**" | "**-**" +
_dec-seq_ = _see-integer-literal-syntax_
====

.Examples
`>>>` [blue]`1.0` +
[green]`1`

`>>>` [blue]`0.123` +
[green]`0.123`

`>>>` [blue]`4.5e+3` +
[green]`4500`

`>>>` [blue]`4.5E-33` +
[green]`4.5e-33`

`>>>` [blue]`4.5E-3` +
[green]`0.0045`

`>>>` [blue]`4.5E10` +
[green]`4.5e+10`


.Arithmetic operators
[cols="^1,^2,6,4"]
|===
| Symbol    | Operation     | Description               | Examples
| [blue]`+` | _Sum_         | Add two values<<note_float_plus_string,^*(1)*^>>            | [blue]`4 + 0.5` -> 4.5
| [blue]`-` | _Subtraction_ | Subtract the right value from the left one | [blue]`4 - 0.5` -> 3.5
| [blue]`*` | _Product_     | Multiply two values       | [blue]`4 * 0.5` -> 2.0
| [blue]`/` | _Float division_    | Divide the left value by the right one | [blue]`1.0 / 2` -> 0.5
| [blue]`./`| _Forced float division_ | Force float division   | [blue]`-1 ./ 2` -> -0.5
|===

====
[[note_float_plus_string]]
^*(1)*^ The sum operator [blue]`+` also supports adding a float number to a string. In this case, the number is converted to a string and prependend or appended to the string, e.g. `"x" + 1.2` results in `"x1.2"`.
====

==== Fractions
_Expr_ also supports fractions. Fraction literals are made with two integers separated by a colon character `:`.

.Fraction literal syntax
====
*_fraction_* = [__sign__] (_num-den-spec_ | _float-spec_) +
_sign_ = "**+**" | "**-**" +
_num-den-spec_ = _digit-seq_ "**:**" _digit-seq_ +
_float-spec_ = _dec-seq_ "**.**" [_dec-seq_] "**(**" _repeated-digits_ "**)**" +
_repeated-digits_ = _dec-seq_ +
_dec-seq_ = _see-integer-literal-syntax_ +
_digit-seq_ = _see-integer-literal-syntax_
====

.Examples
`>>>` [blue]`1 : 2` +
[green]`1:2`

`>>>` [blue]`4:6`  [gray]_// Fractions are always reduced to their lowest terms_ +
[green]`2:3`

`>>>` [blue]`1:2 + 2:3` +
[green]`7:6`

`>>>` [blue]`1:2 * 2:3` +
[green]`1:3`

`>>>` [blue]`1:2 / 1:3` +
[green]`3:2`

`>>>` [blue]`1:2 ./ 1:3` [gray]_// Force decimal division_ +
[green]`1.5`

`>>>` [blue]`-1:2` +
[green]`-1:2`

// `>>>` [blue]`1:-2` [gray]_// Invalid sign specification_ +
// [red]_Eval Error: [1:3] infix operator ":" requires two non-nil operands, got 1_

`>>>` [blue]`1:-2` +
[green]`-1:2`

`>>>` [blue]`1:(-2)` +
[green]`-1:2`

`>>>` [blue]`1.(3)` [gray]_// 1.33333..._ +
[green]`4:3`

`>>>` [blue]`1.2()` [gray]_// parenthesis force float to fraction conversion_ +
[green]`6:5`

`>>>` [blue]`float(6:5)` [gray]_// convert fraction to float_<<note_fraction_to_float,^*(1)*^>> +
[green]`1.2`

`>>>` [blue]`float(1.2())` [gray]_// convert fraction to float_<<note_fraction_to_float,^*(1)*^>> +
[green]`1.2`

====
[[note_fraction_to_float]]
^*(1)*^ The _float()_ function belongs to the builtin-in module "base". Before to use it, you needs to load the module with `BUILTIN "base"`.
====

Fractions can be used in conjunction with integers and floats in expressions.

.Examples
`>>>` [blue]`1:2 + 5` +
[green]`11:2`

`>>>` [blue]`4 - 1:2` +
[green]`7:2`

`>>>` [blue]`1.0 + 1:2` +
[green]`1.5`



=== Strings
Strings are character sequences enclosed in single [blue]`'` or double [blue]`"` quotes.

.Examples
`>>>` [blue]`"I'm a string"` +
[green]`I'm a string`

`>>>` [blue]`"123abc?!"` +
[green]`123abc?!`

`>>>` [blue]`"123\nabc"` +
[green]`123` +
[green]`abc`

`>>>` [blue]`"123\tabc"` +
[green]`123{nbsp}{nbsp}{nbsp}{nbsp}abc`

Some arithmetic operators also apply to strings.

.String operators
[cols="^1,^2,6,4"]
|===
| Symbol    | Operation     | Description               | Examples

| [blue]`+` | _concatenation_ | Join two strings or two _stringable_ values | [blue]`"one" + "two"` -> _"onetwo"_ +
[blue]`"one" + 2` -> _"one2"_

| [blue]`*` | _repeat_ | Make _n_ copy of a string  | [blue]`"one" * 2` -> _"oneone"_
|===

The characters in a string can be accessed by specifying the index using the square `[]` operator. Positive indexes start from the left, with the first character at index 0. Negative indexes start from the right, with the last character at index -1.

.Item access syntax
====
*_item_* = _string-expr_ "**[**" _integer-expr_ "**]**"
====

.Sub-string syntax
====
*_sub-string_* = _string-expr_ "**[**" _integer-expr_ "**:**" _integer-expr_ "**]**"
====

.String examples
`>>>` [blue]`s="abcd"`  [gray]_// assign the string to variable s_ +
[green]`"abcd"`

`>>>` [blue]`s[1]`      [gray]_// char at position 1 (starting from 0)_ +
[green]`"b"`

`>>>` [blue]`s[-1]`    [gray]_// char at position -1, the rightmost one_ +
[green]`"d"`

`>>>` [blue]`#s`       [gray]_// number of chars_ +
[green]`4`

`>>>` [blue]`#"abc"`    [gray]_// number of chars_ +
[green]`3`

`>>>` [blue]`s[1:3]`    [gray]_// chars from position 1 to position 3 excluded_ +
[grean]`"bc"`

=== Booleans
Boolean data type has two values only: [blue]_true_ and [blue]_false_. Relational and boolean expressions result in boolean values.

.Relational operators^(*)^
[cols="^1,^2,6,4"]
|===
| Symbol    | Operation     | Description               | Examples

| [blue]`==` | _Equal_ | True if the left value is equal to the right one | [blue]`5 == 2` -> _false_ +
[blue]`"a" == "a"` -> _true_
| [blue]`!=` | _Not Equal_ | True if the left value is NOT equal to the right one | [blue]`5 != 2` -> _true_ +
[blue]`"a" != "a"` -> _false_
| [blue]`<` | _Less_ | True if the left value is less than the right one | [blue]`5 < 2` -> _false_ +
[blue]`"a" < "b"` -> _true_
| [blue]`\<=` | _Less or Equal_ | True if the left value is less than or equal to the right one | [blue]`5 \<= 2` -> _false_ +
[blue]`"b" \<= "b"` -> _true_
| [blue]`>` | _Greater_ | True if the left value is greater than the right one | [blue]`5 > 2` -> _true_ +
[blue]`"a" > "b"` -> _false_
| [blue]`>=` | _Greater or Equal_ | True if the left value is greater than or equal to the right one | [blue]`5 >= 2` -> _true_ +
[blue]`"b" >= "b"` -> _true_
|===

^(*)^ See also the  [blue]`in` operator in the _array_, _list_ and _dictionary_ sections.


.Boolean operators
[cols="^2,^2,5,4"]
|===
| Symbol    | Operation     | Description               | Examples

| [blue]`NOT` | _Not_ | True if the right value is false | [blue]`NOT true` -> _false_ +
[blue]`NOT (2 < 1)` -> _true_

| [blue]`AND` / [blue]`&&` | _And_ | True if both left and right values are true  | [blue]`false && true` -> _false_ +
[blue]`"a" < "b" AND NOT (2 < 1)` -> _true_

| [blue]`OR` / [blue]`\|\|` | _Or_ | True if at least one of the left and right values integers is true| [blue]`false or true` -> _true_ +
[blue]`"a" == "b" OR (2 == 1)` -> _false_
|===

[CAUTION]
====
Currently, boolean operations are evaluated using _short cut evaluation_. This means that, if the left expression of the [blue]`and` and [blue]`or` operators is sufficient to establish the result of the whole operation, the right expression would not be evaluated at all.

.Example
[source,go]
----
2 > (a=1) or (a=8) > 0; a // <1>
----
<1> This multi-expression returns _1_ because in the first expression the left value of [blue]`or` is _true_ and, as a consequence, its right value is not computed. Therefore the _a_ variable only receives the integer _1_.


TIP: `ecli` provides the _ctrl()_ function that allows to change this behaviour.
====

=== Intervals
_Expr_ supports intervals, which are a special kind of data type. An interval is a range *_open on the right_* of integer values between a lower and an upper bound. The lower and upper bounds can be specified in any order, i.e. the lower bound can be greater than the upper bound. In this case, the interval is considered _descending_. 

Unlike arrays, intervals are not collections of values, but rather a compact representation of a range of values. And unlike other programming languages that separate lower and upper bounds by a colonn, _Expr_ separates them by a double dot symbol `..`.

NOTE: Intervals are _open on the right_, which means that the upper bound is not included in the interval. For example, the interval `1..5` contains the values 1, 2, 3, and 4, but not 5.

_Expr_'s intervals also support step values, which are optionally specified by a third integer value after the upper bound, separated by another doble dot symbol `..`. In this case, the interval represents a range of values starting from the lower bound, up to the upper bound, with a step increment equal to the specified step value. If the step value is not specified, it defaults to 1.

IMPORTANT: The step value must be a positive integer, despite the fact that the lower bound is greater than the upper bound. If it is negative or zero, an error is returned.

.Interval literal syntax
====
*_interval_* = [_lower-bound_] "**..**" _upper-bound_ ["**..**" _step_]

_lower-bound_ = _integer-expr_ +
_upper-bound_ = _integer-expr_ +
_step_ = _positive-integer-expr_
====

.Examples
`>>>` [blue]`3..5` +
[green]`3..5..1`

`>>>` [blue]`5..3` +
[green]`5..3..1`

`>>>` [blue]`..4` +
[green]`0..4..1`

`>>>` [blue]`4..` +
[green]`4..4..1`

`>>>` [blue]`-1..-4` +
[green]`-1..-4..1`

`>>>` [blue]`-1..-4..-1` +
[red]`Eval Error: [1:8] invalid interval specification: step must be positive`

`>>>` [blue]`r = -1..-4` [gray]_// assign interval to variable r_ +
[green]`-1..-4..1`

Main usages of intervals are to generate slices of arrays and to check if a value is in an interval.

=== Arrays
_Expr_ supports arrays of mixed-type values, also specified by normal expressions. Internally, _Expr_'s arrays are Go slices.

.Array literal syntax
====
*_array_* = _empty-array_ | _non-empty-array_ +
_empty-array_ = "**[]**" +
_non-empty-array_ = "**[**" _any-value_ {"**,**" _any-value_} "**]**"
====

.Examples
`>>>` [blue]`[1,2,3]`                     [gray]_// Array of integers_ +
[green]`[1, 2, 3]`

`>>>` [blue]`["one", "two", "three"]`     [gray]_// Array of strings_ +
[green]`["one", "two", "three"]`

`>>>` [blue]`["one", 2, false, 4.1]`      [gray]_// Array of mixed-types_ +
[green]`["one", 2, false, 4.1]`

`>>>` [blue]`["one"+1, 2.0*(9-2)]`        [gray]_// Items as results of expressions_ +
[green]`["one1", 14]`

`>>>` [blue]`[ [1,"one"], [2,"two"]]`     [gray]_// Array of arrays_ +
[green]`[[1, "one"], [2, "two"]]`

.Array operators
[cols="^2,^2,5,4"]
|===
| Symbol    | Operation     | Description               | Examples

| [blue]`+` | _Join_ | Joins two arrays | [blue]`[1,2] + [3]` -> _[1,2,3]_
| [blue]`-` | _Difference_ | Left array without elements in the right array  | [blue]`[1,2,3] - [2]` -> _[1,3]_
| [blue]`>>` +
[blue]`+>` | _Front insertion_ | Insert an item in front | [blue]`0 >> [1,2]` -> _[0,1,2]_
| [blue]`<<` +
[blue]`<+` | _Back insertion_ | Insert an item at end | [blue]`[1,2] << 3` -> _[1,2,3]_
| [blue]`[]` | _Item at index_ | Item at given position | [blue]`[1,2,3][1]` -> _2_
| [blue]`in` | _Item in array_ | True if item is in array | [blue]`2 in [1,2,3]` -> _true_ +
| [blue]`at` | _Index of an item_ | Returns the index of an item in array | [blue]`2 at in [1,2,3]` -> _1_ +
[blue]`6 at [1,2,3]` -> _-1_
| [blue]`#` | _Size_ | Number of items in a array | [blue]`#[1,2,3]` -> _3_
|===

Array's items can be accessed using the index `[]` operator.

.Item access syntax
====
*_item_* = _array-expr_ "**[**" _integer-expr_ "**]**"
====

.Sub-array (or slice of array) syntax
====
*_slice_* = _array-expr_ "**[**" _interval-expr_ "**]**"
====

.Examples: Getting items from arrays
`>>>` [blue]`[1,2,3][1]` +
[green]`2`

`>>>` [blue]`index=2; ["a", "b", "c", "d"][index]` +
[green]`c`

`>>>` [blue]`["a", "b", "c", "d"][2..4]` +
[green]`["c", "d"]`

`>>>` [blue]`array=[1,2,3]; array[1]` +
[green]`2`

`>>>` [blue]`["one","two","three"][1]` +
[green]`two`

`>>>` [blue]`array=["one","two","three"]; array[2-1]` +
[green]`two`

`>>>` [blue]`array[1]="six"; array` +
[green]`["one", "six", "three"]`

`>>>` [blue]`array[-1]` +
[green]`three`

`>>>` [blue]`array[10]` +
[red]`Eval Error: [1:9] index 10 out of bounds`

.Example: Number of elements in an array
`>>>` [blue]`#array` +
[green]`3`

.Examples: Element insertion
`>>>` [blue]`"first" >> array` +
[green]`["first", "one", "six", "three"]`

`>>>` [blue]`array << "last"` +
[green]`["first", "one", "six", "three", "last"]`

.Examples: Element in array
`>>>` [blue]`"six" in array` +
[green]`true`

`>>>` [blue]`"ten" in array` +
[green]`false`

.Examples: Index of an element in array
`>>>` [blue]`"six" at array` +
[green]`2`

`>>>` [blue]`"ten" at array` +
[green]`-1`

.Examples: Concatenation and filtering
`>>>` [blue]`[1,2,3] + ["one", "two", "three"]` +
[green]`[1, 2, 3, "one", "two", "three"]`

`>>>` [blue]`[1,2,3,2,4] - [2,4]` [gray]_Note that *all* occurrences are removed_ +
[green]`[1, 3]`

=== Linked Lists
A linked list is a fundamental linear data structure in computer science. Unlike an array, where elements are stored in contiguous memory locations, the elements in a linked list are scattered in memory.

These elements are called _nodes_. Each node consists of two main parts:

_Data_: The actual value or information stored in the node. +
_Pointer_ (or _Reference_): A link that points to the memory address of the next node in the sequence.

The first node in the list is called the _head_. The last node in the list is called the _tail_. 

Internally, Expr's linked lists hold two pointers: one to the head of the list and one to the tail of the list. This allows for efficient insertion and deletion of items at both ends of the list. Also, linked lists keep track of their size, so the number of items in a linked list can be obtained in constant time.

.Linked list literal syntax
====
*_linked-list_* = "**[<**" [_item-expr_ {"**,**" _item-expr_}] "**>]**" +
_item-expr_ = _any-value_
====

`>>>` [blue]`ls=[<1,2,3,4>]` +
[green]`[<1, 2, 3, 4>]`

In general, arrays and linked-lists support similar operations. However, there are some differences between them. For example, arrays support efficient random access to elements, while linked-lists support efficient insertion and deletion of elements at both ends of the list.


#todo: to be completed#

=== Dictionaries
The _dictionary_, or _dict_, data-type represents sets  of pairs _key/value_. It is also known as _map_ or _associative array_.

Dictionary literals are sequences of pairs separated by comma [blue]`,` enclosed in curly brackets [blue]`{}`.

.Dict literal syntax
====
*_dict_* = _empty-dict_ | _non-empty-dict_ +
_empty-dict_ = "**{}**" +
_non-empty-dict_ = "**{**" _key-scalar_ "**:**" _any-value_ {"**,**" _key-scalar_ "**:**" _any-value_} "**}**" <<note_scalar_key,^*(1)*^>> +
====

====
[[note_scalar_key]]
^*(1)*^ Currently, _key-scalar_ type can only be _integer_ or _string_.
====

To get the value of a dictionary's item, the key must be specified. The key is used to access the corresponding value in the dictionary.
Two access methods are available: the _key in square brackets_ and the _dot notation_ method.

.Item access syntax by key in square brackets
====
*_item_* = _dict-expr_ "**[**" _key-expr_ "**]**"
====

.Item access syntax by dot notation
====
*_item_* = _dict-expr_ "**.**" _key-expr_
====

.Dict operators
[cols="^2,^2,4,5"]
|===
| Symbol    | Operation     | Description               | Examples

| [blue]`+` | _Join_ | Joins two dicts | [blue]`{1:"one"} + {6:"six"}` -> _{1: "one", 6: "six"}_
| [blue]`[]` | _Dict item value_ | Item value of given key | [blue]`{"one":1, "two":2}["two"]` -> _2_
| [blue]`.` | _Dict item value_ | Item value of given key | [blue]`{"one":1, "two":2}."two"` -> _2_
| [blue]`in` | _Key in dict_ | True if key is in dict | [blue]`"one" in {"one":1, "two":2}` -> _true_ +
| [blue]`at` | _Key of given value_ | Returns the key of given value | [blue]`1 at {"one":1, "two":2}` -> _"one"_ +
[blue]`6 at {"one":1, "two":2}` -> _nil_
| [blue]`#` | _Size_ | Number of items in a dict | [blue]`#{1:"a", 2:"b", 3:"c"}` -> _3_
|===

.Examples
`>>>` [blue]`{1:"one", 2:"two"}` +
[green]`{1: "one", 2: "two"}`

`>>>` [blue]`{"one":1, "two": 2}` +
[green]`{"one": 1, "two": 2}`

`>>>` [blue]`{"sum":1+2+3, "prod":1*2*3}` +
[green]`{"sum": 6, "prod": 6}`

`>>>` [blue]`{"one":1, "two":2}["two"]` +
[green]`2`

`>>>` [blue]`{"one":1, "two":2}."two"` <<note_dict_dot_notation,^*(1)*^>> +
[green]`2`

`>>>` [blue]`d={"one":1, "two":2}; d["six"]=6; d` +
[green]`{"two": 2, "one": 1, "six": 6}`

`>>>` [blue]`#d` +
[green]`3`


====
[[note_dict_dot_notation]]
^*(1)*^ When the key type is string, the key value can be expressed without quotes in the dot notation method.

Example: `d={"one":1, "two":2}; d.two` returns _2_.

To force the evaluation of a key expression, enclose it in parenthesis. Example:

`a="b"; d={"a":"one", "b":"two"}; d.(a)` returns _"two"_.

whereas

`a="b"; d={"a":"one", "b":"two"}; d.a` returns _"one"_.
====

== Variables
_Expr_, like most programming languages, supports variables. A variable is an identifier with an assigned value. Variables are stored in _contexts_.

.Variable literal syntax
====
*_variable_* = _identifier_ "*=*" _any-value_ +
_identifier_ = _alpha_ {_alpha_|_dec-digit_|"*_*"} +
__alpha__ = "*a*"|"*b*"|..."*z*"|"*A*"|"*B*"|..."*Z*"
====

NOTE: The assign operator <<section_assignment_operators,[blue]`=`>> returns the value assigned to the variable.

.Examples
`>>>` [blue]`a=1` +
[green]`1`

`>>>` [blue]`a_b=1+2` +
[green]`3`

`>>>` [blue]`a_b` +
[green]`3`

`>>>` [blue]`x = 5.2 * (9-3)`  [gray]_// The assigned value here has the typical approximation error of the float data-type_ +
[green]`31.200000000000003`

`>>>` [blue]`x = 1; y = 2*x` +
[green]`2`

`>>>` [blue]`\_a=2` +
[red]`Parse Error: [1:2] unexpected token "_"`

`>>>` [blue]`1=2` +
[red]`Eval Error: [1:2] left operand of "=" must be a variable or a collection's item`


== Other operations

=== [blue]`;` operator
The semicolon operator [blue]`;` is an infixed pseudo-operator. It evaluates the left expression first and then the right expression. The value of the latter is the final result.

.Multi-expression syntax
====
*_multi-expression_* = _expression_ {"**;**" _expression_ }
====

An expression that contains [blue]`;` is called a _multi-expression_ and each component expression is called a _sub-expression_.

IMPORTANT: Technically [blue]`;` is not treated as a real operator. It acts as a separator in lists of expressions.

TIP: [blue]`;` can be used to set some variables before the final calculation.

.Example
`>>>` [blue]`a=1; b=2; c=3; a+b+c` +
[green]`6`

The value of each sub-expression is stored in the automatic variable _last_.

.Example
`>>>` [blue]`2+3; b=last+10; last` +
[green]`15`


=== [blue]`but` operator
[blue]`but` is an infixed operator. Its operands can be expressions of any type. It evaluates the left expression first, then the right expression. The value of the right expression is the final result.

.Examples
`>>>` [blue]`5 but 2` +
[green]`2`

`>>>` [blue]`x=2*3 but x-1` +
[green]`5`

[blue]`but` behavior is very similar to [blue]`;`. The only difference is that [blue]`;` is a pseudo-operator and can't be used inside parenthesis [blue]`(` and [blue]`)`. Furthermore, [blue]`but`, being a true operator, does not set the _last_ variable.

[[section_assignment_operators]]
=== Assignment operators [blue]`=` and [blue]`:=`
The assignment operator [blue]`=` is used to define variables or to change their value in the evaluation context (see _ExprContext_).

The value on the left side of [blue]`=` must be a variable identifier or an expression that evalutes to a variable. The value on the right side can be any expression and its value becomes the result of the assignment operation.

.Examples
`>>>` [blue]`a=15+1` +
[green]`16`

`>>>` [blue]`L=[1,2,3]; L[1]=5; L` +
[green]`[1, 5, 3]`

The [blue]`:=` operator has same effect as [blue]`=` for scalar values. However their behavior is different for array, linked-list and dict variables. In such cases [blue]`=` assignes a reference to the variable, whereas [blue]`:=` assignes a deep copy of the value.

.Examples
[gray]_// Assign reference with '='_ +
`>>>` [blue]`L=[1,2,3]; M=L; M[0]=42; L` +
[green]`[42, 2, 3]` +
[gray]_// In this case, both L and M point to the same array. Therefore, changing M also changes L._


[gray]_// Assign deep copy with ':='_ +
`>>>` [blue]`L=[1,2,3]; M:=L; M[0]=42; L` +
[green]`[1, 2, 3]` +
[gray]_// In this case, L and M are separate arrays. Therefore, changing M does not affect L._


=== Selector operator [blue]`? : ::`
The _selector operator_ is very similar to the _switch/case/default_ statement available in many programming languages.

.Selector literal syntax
====
*_selector-operator_* = _select-expression_ "*?*" _selector-case_ { "*:*" _selector-case_ } ["*::*" _default-multi-expression_]

_selector-case_ = [_match-list_] _case-multi-expression_ +
_match-list_ = "*[*" _item_ {"*,*" _items_} "*]*" +
_item_ = _expression_ +
_case-multi-expression_ = "*{*" _multi-expression_ "*}*" +
_multi-expression_ = _expression_ { "*;*" _expression_ } +
_default-multi-expression_ = _multi-expression_
====

In other words, the selector operator evaluates the _select-expression_ on the left-hand side of the [blue]`?` symbol; it then compares the result obtained with the values listed in the __match-list__, from left to right. If the comparision finds a match with a value in a _match-list_, the associated _case-multi-expression_ is evaluted, and its result will be the final result of the selection operation.

The match lists are optional. In that case, the position, from left to right, of the _selector-case_ is used as _match-list_. Of course, that only works if the _select-expression_ results in an integer.

The [blue]`:` symbol (colon) is the separator of the selector-cases. Note that if the value of the _select-expression_ does not match any _match-list_, an error will be issued. Therefore, it is strongly recommended to provide a default (multi-)expression introduced by the [blue]`::` symbol (double-colon). Also note that the default expression has no _match-list_.

.Examples
`>>>` [blue]`1 ? {"a"} : {"b"}` +
[green]`b`

`>>>` [blue]`10 ? {"a"} : {"b"} {2colon} {"c"}` +
[green]`c`

`>>>` [blue]`10 ? {"a"} :[true, 2+8] {"b"} {2colon} {"c"}` +
[green]`b`

`>>>` [blue]`10 ? {"a"} :[true, 2+8] {"b"} {2colon} [10] {"c"}` +
[red]`Parse Error: [1:34] case list in default clause`

`>>>` [blue]`10 ? {"a"} :[10] {x="b" but x} {2colon} {"c"}` +
[green]`b`

`>>>` [blue]`10 ? {"a"} :[10] {x="b"; x} {2colon} {"c"}` +
[green]`b`

`>>>` [blue]`11 ? {"a"} :[10] {"b"} {2colon} {"c"}` +
[green]`c`

`>>>` [blue]`10 ? {"a"} : {"b"}` +
[red]`Eval Error: [1:3] no case catches the value (10) of the selection expression`

==== Triple special case of the selector operator
If the _select-expression_ is a boolean expression, the selector operator can be used as a sort of _if-then-else_ statement. In this case, the first case is evaluated if the _select-expression_ is true, and the second case is evaluated if the _select-expression_ is false. In this special case, the _match-list_ of both cases must be empty.

.Example
`>>>` [blue]`(true) ? {"T"}: {"F"}` +
[green]`T` +
`>>>` [blue]`(2 > 1) ? {"a"} : {"b"}` +
[green]`a` +
`>>>` [blue]`(2 < 1) ? {"a"} : {"b"}` +
[green]`b`

[WARNING]
====
The triple special case of the selector operator is very useful, but it only works with boolean expressions.

.Example of confusion
`>>>` [blue]`int(true) ? {"T"}: {"F"}` +
[green]`F`
====

=== Variable default value [blue]`??`, [blue]`?=`, and [blue]`?!`
The left operand of the first two operators, [blue]`??` and [blue]`?=`, must be a variable. The right operand can be any expression. They return the value of the variable if this is defined; otherwise they return the value of the right expression.

IMPORTANT: If the left variable is defined, the right expression is not evaluated at all.

The [blue]`??` operator do not change the status of the left variable.

The [blue]`?=` assigns the calculated value of the right expression to the variable on the left side.

The third one, [blue]`?!`, is the alternate operator. If the variable on the left side is not defined, it returns [blue]_nil_. Otherwise it returns the result of the expression on the right side.

IMPORTANT: If the variable [blue]`?!` is NOT defined, the expression is not evaluated at all.

.Examples: effect of the [blue]`??` operator
`>>>` [blue]`var ?? (1+2)` +
[green]`3`

`>>>` [blue]`var` +
[red]`Eval Error: undefined variable or function "var"`

.Examples: effect of the [blue]`?=` operator
`>>>` [blue]`var ?= (1+2)` +
[green]`3`

`>>>` [blue]`var` +
[green]`3`

`>>>` [blue]`var ?= 5` +
[green]`3`

`>>>` [blue]`var` +
[green]`3`

.Examples: effect of the [blue]`?!` operator
`>>>` [blue]`x ?! 5` +
[green]`nil`

`>>>` [blue]`x=1; x ?! 5` +
[green]`5`

`>>>` [blue]`y ?! (c=5); c` +
[red]`Eval Error: undefined variable or function "c"`

NOTE: These operators have a high priority, in particular higher than the operator [blue]`=`.

== Priorities of operators
The table below shows all supported operators by decreasing priorities.

.Operators priorities (in this table "list" means _array_ or _linked-list_)
[cols="^3,^2,^2,^5,^6"]
|===
| Priority | Operator     | Position | Operation | Operands and results

.2+|*ITEM*| [blue]`[`...`]` | _Postfix_ | _List item_| _list_ `[`  _integer_ `]` -> _any_
| [blue]`[`...`]` | _Postfix_ | _Dict item_ | _dict_ `[` _any_ `]` -> _any_
.5+|*INC/DEC*| [blue]`++` | _Postfix_ | _Iterator next item_ | _iterator_ `++` -> _any_
| [blue]`++` | _Postfix_ | _Post increment_| _integer-variable_ `++` -> _integer_
| [blue]`++` | _Prefix_ | _Pre increment_ | `++` _integer-variable_ -> _integer_
| [blue]`--` | _Postfix_ | _Post decrement_ | _integer-variable_ `--` -> _integer_
| [blue]`--` | _Prefix_ | _Pre decrement_ | `--` _integer-variable_ -> _integer_
.3+|*DEFAULT*| [blue]`??` | _Infix_ | _Default value_| _variable_ `??` _any-expr_ -> _any_
| [blue]`?=` | _Infix_ | _Default/assign value_| _variable_ `?=` _any-expr_ -> _any_
| [blue]`?!` | _Infix_ | _Alternate value_| _variable_ `?!` _any-expr_ -> _any_
//.1+| *ITER*^1^| [blue]`()` | _Prefix_ | _Iterator value_ | `()` _iterator_ -> _any_
.1+|*FACT*| [blue]`!` | _Postfix_ | _Factorial_| _integer_ `!` -> _integer_
.3+|*SIGN*| [blue]`+`, [blue]`-` | _Prefix_ | _Change-sign_| (`+`\|`-`) _number_ -> _number_
| [blue]`#` | _Prefix_ | _Lenght-of_ | `#` _collection_ -> _integer_
| [blue]`#` | _Prefix_ | _Size-of_ | `#` _iterator_ -> _integer_
.4+|*BIT SHIFT* +
and
*INSERT*| [blue]`<<` | _Infix_ | _Left-Shift_ | _integer_ `<<` _integer_ -> _integer_
| [blue]`>>` | _Infix_ | _Right-Shift_ | _integer_ `>>` _iterator_ -> _integer_
| [blue]`>>` | _Infix_ | _Head-Insert_ | _any_ `>>` _list_ -> _list_
| [blue]`<<` | _Infix_ | _Tail-Insert_ | _list_ `<<` _any_ -> _list_
.3+|*SELECT*| [blue]`? : ::` | _Multi-Infix_ | _Case-Selector_ | _any-expr_ `?` _case-list_ _case-expr_ `:` _case-list_ _case-expr_ ... `::` _default-expr_ -> _any_
| [blue]`? : ::` | _Multi-Infix_ | _Index-Selector_ | _int-expr_ `?` _case-expr_ `:` _case-expr_ ... `::` _default-expr_ -> _any_
| [blue]`? :` | _Multi-Infix_ | _Boolean-Selector_ +
(aka _triple_)| _bool-expr_ `?` _true-case-expr_ `:` _false_case-expr_ -> _any_
.1+|*FRACT*| [blue]`:` | _Infix_ | _Fraction_ | _integer_ `:` _integer_ -> _fraction_
.7+|*PROD*| [blue]`*` | _Infix_ | _Product_ | _number_ `*` _number_ -> _number_
| [blue]`*` | _Infix_ | _String-repeat_ | _string_ `*` _integer_ -> _string_
| [blue]`/` | _Infix_ | _Division_ | _number_ `/` _number_ -> _number_
| [blue]`./` | _Infix_ | _Float-division_ | __number__ `./` _number_ -> _float_
| [blue]`/` | _Infix_ | _Split_ | _string_ `/` _string_ -> _list_
| [blue]`/` | _Infix_ | _Split_ | _string_ `/` integer -> _list_
| [blue]`%` | _Infix_ | _Division-remainder_ | _integer_ `%` _integer_ -> _integer_
.6+|*SUM*| [blue]`+` | _Infix_ | _Sum_ | _number_ `+` _number_ -> _number_
| [blue]`+` | _Infix_ |  _String-concat_ | (_string_\|_number_) `+` (_string_\|_number_) -> _string_
| [blue]`+` | _Infix_ |  _List-join_ | _list_ `+` _list_ -> _list_
| [blue]`+` | _Infix_ |  _Dict-join_ | _dict_ `+` _dict_ -> _dict_
| [blue]`-` | _Infix_ | _Subtraction_ | _number_ `-` _number_ -> _number_
| [blue]`-` | _Infix_ | _List-difference_ | _list_ `-` _list_ -> _list_
.1+|*BITWISE NOT*| [blue]`~` | _Prefix_ | _Binary Not_ | `~` _integer_ -> _integer_
.1+|*BITWISE AND*| [blue]`&` | _Infix_ | _Binary And_ | _integer_ `&` _integer_ -> _integer_
.2+|*BITWISE OR*| [blue]`\|` | _Infix_ | _Binary Or_ | _integer_ `\|` _integer_ -> _integer_
| [blue]`^` | _Infix_ | _Binary Xor_ | _integer_ `^` _integer_ -> _integer_
.10+|*RELATION*| [blue]`<` | _Infix_ | _Less_ | _comparable_ `<` _comparable_ -> _boolean_
| [blue]`\<=` | _Infix_ |  _less-equal_ | _comparable_ `\<=` _comparable_ -> _boolean_
| [blue]`>` | _Infix_ |  _Greater_ | _comparable_ `>` _comparable_ -> _boolean_
| [blue]`>=` | _Infix_ |  _Greater-equal_ | _comparable_ `>=` _comparable_ -> _boolean_
| [blue]`==` | _Infix_ |  _Equal_ | _comparable_ `==` _comparable_ -> _boolean_
| [blue]`!=` | _Infix_ |  _Not-equal_ | _comparable_ `!=` _comparable_ -> _boolean_
| [blue]`in` | _Infix_ |  _Member-of-list_ | _any_ `in` _list_ -> _boolean_
| [blue]`in` | _Infix_ |  _Key-of-dict_ | _any_ `in` _dict_ -> _boolean_
| [blue]`at` | _Infix_ |  _Index-of-value-in-list_ | _any_ `at` _dict_ -> _intger_
| [blue]`at` | _Infix_ |  _Key-of-value-in-dict_ | _any_ `at` _dict_ -> _key_
.1+|*LOGIC NOT*| [blue]`not` | _Prefix_ | _Not_ | `not` _boolean_ -> _boolean_
.2+|*LOGIC AND*| [blue]`and` | _Infix_ | _And_ | _boolean_ `and` _boolean_ -> _boolean_
| [blue]`&&` | _Infix_ | _And_ | _boolean_ `&&` _boolean_ -> _boolean_
.2+|*LOGIC OR*| [blue]`or` | _Infix_ | _Or_ | _boolean_ `or` _boolean_ -> _boolean_
| [blue]`\|\|` | _Infix_ | _Or_ | _boolean_ `\|\|` _boolean_ -> _boolean_
.2+|*INSERT*| [blue]`+>` | _Infix_ | _Prepend_ | _any_ `+>` _list_ -> _list_
| [blue]`<+` | _Infix_ | _Append_ | _list_ `<+` _any_ -> _list_
.3+|*ASSIGN*| [blue]`=` | _Infix_ | _Ref-Assignment_ | _identifier_ `=` _any_ -> _any_
| [blue]`:=` | _Infix_ | _Copy-Assignment_ | _identifier_ `=` _any_ -> _any_
4+| _See also the table of special assignment operators below_
.1+|*BUT*| [blue]`but` | _Infix_ | _But_ | _any_ `but` _any_ -> _any_
.6+|*ITER-OP*| [blue]`digest` | _Infix_ | _Item-digesting_ | _iterable_ `digest` _expr_ -> _any_
| [blue]`filter` | _Infix_ | _Item-filtering_ | _iterable_ `filter` _expr_ -> _iterable_
| [blue]`groupby` | _Infix_ | _Dict-grouping_ | _iterable_ `groupby` _key-expr_ -> _dict_
| [blue]`cat` | _Infix_ | _Item-concatenation_ | _iterable_ `cat` _iterable_ -> _iterable_
| [blue]`map` | _Infix_ | _Item-mapping_ | _iterable_ `map` _expr_ -> _iterable_
4+| _See iterators section for examples_
.1+|*RANGE*| [blue]`:` | _Infix_ | _Index-range_ | _integer_ `:` _integer_ -> _integer-pair_
|===

//^1^ Experimental

.Special assignment operators
[cols="^2,^2,^4,^6"]
|===
| Priority | Operator    | Operation |Equivalent operation

.9+|*ASSIGN*| [blue]`+=` | _Sum & Assign_ | _var_ `\+=` _expr_  +
short for +
_var_ `=` _value-of-var_ `+` _expr_
| [blue]`-=` | _Subtract & Assign_ | _var_ `-=` _expr_  +
short for +
_var_ `=` _value-of-var_ `-` _expr_
| [blue]`*=` | _Multiply & Assign_ | _var_ `\*=` _expr_  +
short for +
_var_ `=` _value-of-var_ `*` _expr_
| [blue]`/=` | _Divide & Assign_ | _var_ `/=` _expr_  +
short for +
_var_ `=` _value-of-var_ `/` _expr_
| [blue]`%=` | _Remainder & Assign_ | _var_ `%=` _expr_  +
short for +
_var_ `=` _value-of-var_ `%` _expr_
| [blue]`&=` | _Bitwise and & Assign_ | _var_ `&=` _expr_  +
short for +
_var_ `=` _value-of-var_ `&` _expr_
| [blue]`\|=` | _Bitwise or & Assign_ | _var_ `\|=` _expr_  +
short for +
_var_ `=` _value-of-var_ `\|` _expr_
| [blue]`<\<=` | _Left shift & Assign_ | _var_ `<\<=` _expr_  +
short for +
_var_ `=` _value-of-var_ `<<` _expr_
| [blue]`>>=` | _Right shift & Assign_ | _var_ `>>=` _expr_  +
short for +
_var_ `=` _value-of-var_ `>>` _expr_
|===

== Functions
Functions in _Expr_ are very similar to functions available in many programming languages. Currently, _Expr_ supports two types of function, _expr-functions_ and _go-functions_.

* _expr-functions_ are defined using _Expr_'s syntax. They can be passed as arguments to other functions and can be returned from functions. Moreover, they bind themselves to the defining context, thus becoming closures.
* _go-functions_ are regular Golang functions callable from _Expr_ expressions. They are defined in Golang source files called _modules_ and compiled within the _Expr_ package. To make Golang functions available in _Expr_ contextes, it is required to activate the builtin module or to load the plugin module in which they are defined.


=== _Expr_ function definition
An expr-function is identified and referenced by its name. It can have zero or more parameter. expr-functions also support optional parameters and passing paramters by name.

.Expr's function definition syntax
====
*_function-definition_* = _identifier_ "**=**"  "**func(**" [_formal-param-list_] "**)**"  "**{**" _multi-expression_ "**}**"

_formal-param-list_ = _required-param-list_ [ "**,**" _optional-param-list_ ] +
_required-param-list_ = _identifier_ { "**,**" _identifier_ } +
_optional-param-list_ = _optional-parm_ { "**,**" _optional-param_ } +
_optional-param_ = _param-name_ "**=**" _any-expr_ +
_param-name_ = _identifier_
====

[[ex_sum_function]]
.Examples
`>>>` [gray]_// A simple function: it takes two parameters and returns their "sum"_**^(*)^** +
`>>>` [blue]`sum = func(a, b){ a + b }` +
[green]`sum(a, b):any{}`

^(*)^ Since the plus, **+**, operator is defined for multiple data-types, the _sum()_ function can be used for any pair of that types.

[[ex_fib_function]]
`>>>` [gray]_// A more complex example: recursive calculation of the n-th value of Fibonacci's sequence_ +
`>>>` [blue]`fib = func(n){ n ? [0] {0}: [1] {1} :: {fib(n-1)+fib(n-2)} }` +
[green]`fib(n):any{}`


`>>>` [gray]_// Same function fib() but entered by splitting it over mulple text lines_ +
`>>>` [blue]`fib = func(n){ \` +
`\...` [blue]`{nbsp}{nbsp}n ? \` +
`\...` [blue]`{nbsp}{nbsp}{nbsp}{nbsp}[0] {0} : \` +
`\...` [blue]`{nbsp}{nbsp}{nbsp}{nbsp}[1] {1} :: \` +
`\...` [blue]`{nbsp}{nbsp}{nbsp}{nbsp}{ \` +
`\...` [blue]`{nbsp}{nbsp}{nbsp}{nbsp}{nbsp}{nbsp}fib(n-1) + fib(n-2) \` +
`\...` [blue]`{nbsp}{nbsp}{nbsp}{nbsp}} \` +
`\...` [blue]`}` +
[green]`fib(n):any{}`

[[ex_measure_function]]
`>>>` [gray]_// Required and optional parameters_ +
`>>>` [blue]`measure = func(value, unit="meter"){ value + " " + unit + (value > 1) ? {"s"} :: {""}}` +
[green]`measure(value, unit="meter"):any{}`


=== _Golang_ function definition
Description of how to define Golang functions and how to bind them to _Expr_ are topics covered in other documents.

=== Function calls
To call a function, either Expr or Golang type, it is necessary to specify its name and, at least, its required parameters.

.Function invocation syntax
====
*_function-call_* = _identifier_ "**(**" _actual-param-list_ "**)**"

_actual-param-list_ = [_positional-params_] [_named-parameters_] +
_positional-params_ = _any-value_ { "*,*" _any-value_ } +
_named-params_ = _param-name_ "**=**" _any-value_ { "*,*" _param-name_ "**=**" _any-value_ } +
_param-name_ = _identifier_
====

.Examples of calling the <<ex_sum_function,`sum()`>> functions defined above
`>>>` [gray]_// sum of two integers_ +
`>>>` [blue]`sum(-6, 2)` +
[green]`-4` +
`>>>` [gray]_// same as above but passing the parameters by name_ +
`>>>` [blue]`sum(a=-6, b=2)` +
[green]`-4` +
`>>>` [gray]_// again, but swapping parameter positions (see the diff() examples below)_ +
`>>>` [blue]`sum(b=2, a=-6)` +
[green]`-4` +
`>>>` [gray]_// sum of a fraction and an integer_ +
`>>>` [blue]`sum(3|2, 2)` +
[green]`7|2` +
`>>>` [gray]_// sum of two strings_ +
`>>>` [blue]`sum("bye", "-bye")` +
[green]`"bye-bye"` +
`>>>` [gray]_// sum of two arrays_ +
`>>>` [blue]`sum(["one", 1], ["two", 2])` +
[green]`["one", 1, "two", 2]`

=== Parameters passed by name
When calling a function, it is possible to pass parameters by name. In that case, the order of the parameters does not matter. The only requirement is that all required parameters must be specified.

.Examples of calling a function with parameters passed by name
`>>>` [gray]_// diff(a,b) calculates a-b_ +
`>>>` [blue]`diff = func(a,b){a-b}` +
[green]`diff(a, b):any{}` +
`>>>` [gray]_// simple invocation_ +
`>>>` [blue]`diff(10,8)` +
[green]`2` +
`>>>` [gray]_// swapped parameters passed by name_ +
`>>>` [blue]`diff(b=8,a=10)` +
[green]`2`

[IMPORTANT]
====
In function calls, parameters passed by name must be specified after all positional parameters. Otherwise, a parse error will be issued.
====

.Examples of calling the <<ex_fib_function,`fib()`>> function defined above
`>>>` [gray]_// Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, ..._ +
`>>>` [blue]`fib(6)` +
[green]`8` +
`>>>` [blue]`fib(9)` +
[green]`34`

.Examples of calling the <<ex_measure_function,`measure()`>> functions defined above
`>>>` [gray]_// simple call_ +
`>>>` [blue]`measure(10,"litre")` +
[green]`"10 litres"` +
`>>>` [gray]_// accept the default unit_ +
`>>>` [blue]`measure(8)` +
[green]`"8 meters"` +
`>>>` [gray]_// without the required parameter 'value'_ +
`>>>` [blue]`measure(unit="degrees"))` +
[red]`Eval Error: measure(): missing params -- value`

.Examples of context binding (closures)
`>>>` [blue]`factory = func(n=2){ func(x){x*n} }` +
[green]`factory(n=2):any{}` +
`>>>` [blue]`double = factory()` +
[green]`double(x):any{}` +
`>>>` [blue]`triple = factory(3)` +
[green]`triple(x):any{}` +
`>>>` [blue]`double(5)` +
[green]`10` +
`>>>` [blue]`triple(5)` +
[green]`15`


=== Function context
Functions compute values in a local context (scope) that do not make effects on the calling context. This is the normal behavior. Using the _clone_ modifier [blue]`@` it is possibile to export local definition to the calling context. The clone modifier must be used as prefix to variable names and it is part of the name. E.g. [blue]`@x` is not the same as [blue]`x`; they are two different and unrelated variables.

Clone variables are normal local variables. The only diffence will appear when the defining function ends, just before the destruction of its local context. At that point, all local clone variables are cloned in the calling context with the same names but the [blue]`@` symbol.

.Example
`>>>` [blue]`f = func() { @x = 3; x = 5 }` [gray]_// f() declares two *different* local variables: ``@x`` and ``x``_ +
[green]`f():any{}` +
`>>>` [blue]`f()` [gray]_// The multi-expression (two expressions) in f() is calculated and the last result is returned_ +
[green]`5` +
`>>>` [blue]`x` [gray]_// The `x` variable was not defined in the main context before the f() invocation. It appears in the main context by cloning the `@x` variable, local to f(), after its termnation._ +
[green]`3`

NOTE: The clone modifier [blue]`@` does not make a variable a reference variable, as the ampersand character `&` does in languages such as C and C++. 

[IMPORTANT]
====
The clone modifier can also be used to declare the formal parameters of functions, because they are local variables too.

.Example
`>>>` [blue]`g = func(@p) {2+@p}` +
[green]`g(@p):any{}` +
`>>>` [blue]`g(9)` +
[green]`11` +
`>>>` [blue]`p` +
[green]`9`
====

== Builtins
Builtins are collection of function dedicated to specific domains of application. They are defined in Golang source files called _modules_ and compiled within the _Expr_ package. To make builtins available in _Expr_ contextes, it is required to activate the builtin module in which they are defined.

There are currently several builtin modules. More builtin modules will be added in the future.

.Available builtin modules
* <<_module_base,*base*>>: Base expression tools like isNil(), int(), etc.
* <<_module_fmt,*fmt*>>: String and console formatting functions
* <<_module_import,*import*>>: Functions <<_import,import()>> and <<_include,include()>>
* <<_module_iterator,*iterator*>>: Iterator helper functions
* <<_module_math_arith,*math.arith*>>: Functions <<_add,add()>> and <<_mul,mul()>>
* <<_module_os_file,*os.file*>>: Operating system file functions
* <<_module_string,*string*>>: string utilities

Builtins activation is done by using the [blue]`BUILTIN` operator. All modules except "base" must be explicitly enabled. The syntax is as follows.

.Builtin activation syntax
====
*_builtin-activation_* = [blue]`BUILTIN` (_builtin-name_ | _list-of-builtin-names_ | **"*"**) +
_builtin-name_ = _string_ +
_list-of-builtin-names_ = **[** _string_ { "**,**" _string_ } **]**
====

The following example shows how to activate the builtin module "math.arith" and then use the function add() defined in that module.

.Example: using built functions
`>>>` [blue]`BUILTIN "math.arith"` +
[green]`1` +
`>>>` [blue]`add(5, 3, -2)` +
[green]`6`

TIP: To avoid the need to activate builtin modules one by one, it is possible to activate all builtin modules at once by using the [blue]`BUILTIN "*"` syntax.

=== Builtin modules

==== Module "base"
The "base" builtin module provides functions for type checking and type conversion. These functions are always available in _Expr_ contexts without the need to activate the "base" module.

.Checking functions
* <<_isbool,isBool()>>
* <<_isdict,isDict()>>
* <<_isfloat,isFloat()>>
* <<_isfract,isFract()>>
* <<_isarray,isArray()>>
* <<_islist,isList()>>
* <<_isnil,isNil()>>
* <<_isrational,isRational()>>
* <<_isstring,isString()>>

.Conversion functions
* <<_bool,bool()>>
* <<_int,int()>>
* <<_float,float()>>
* <<_string,string()>>
* <<_fract,fract()>>

.Other functions
* <<_char,char()>>
* <<_eval,eval()>>
* <<_set,set()>>
* <<_var,var()>>


===== isBool()
Syntax: `isBool(<expr>) -> bool` +
Returns _true_ if the value type of _<expr>_ is boolean, false otherwise.

.Examples
`>>>` [blue]`isBool(true)` +
[green]`true` +
`>>>` [blue]`isBool(3==2)` +
[green]`true` +
`>>>` [blue]`isBool(3 + 2)` +
[green]`false`

===== isDict()
Syntax: `isDict(<expr>) -> bool` +
Returns _true_ if the value type of _<expr>_ is dictionary, false otherwise.

.Examples
`>>>` [blue]`isDict({})` +
[green]`true` +
`>>>` [blue]`isDict({1: "one", 2: "two"})` +
[green]`true` +
`>>>` [blue]`isDict(1:"one")` +
[red]`Eval Error: denominator must be integer, got string (one)`

===== isFloat()
Syntax: `isFloat(<expr>) -> bool` +
Returns _true_ if the value type of _<expr>_ is float, false otherwise.

.Examples
`>>>` [blue]`isFloat(4.)` +
[green]`true` +
`>>>` [blue]`isFloat(4)` +
[green]`false` +
`>>>` [blue]`isFloat("2.1")` +
[green]`false`

===== isFract()
Syntax: `isFract(<expr>) -> bool` +
Returns _true_ if the value type of _<expr>_ is fraction, false otherwise.

.Examples
`>>>` [blue]`isFract(4.5)` +
[green]`false` +
`>>>` [blue]`isFract(4:5)` +
[green]`true` +
`>>>` [blue]`isFract(4)` +
[green]**`false`** +
`>>>` [blue]`isFract(1.(3))` +
[green]`true`

===== isArray()
Syntax: `isArray(<expr>) -> bool` +
Returns _true_ if the value type of _<expr>_ is array, false otherwise.

.Examples
`>>>` [blue]`isArray([])` +
[green]`true` +
`>>>` [blue]`isArray([1, "2"])` +
[green]`true` +
`>>>` [blue]`isArray(1,2)` +
[red]`Eval Error: isList(): too many params -- expected 1, got 2`

===== isList()
Syntax: `isList(<expr>) -> bool` +
Returns _true_ if the value type of _<expr>_ is list, false otherwise.

.Examples
`>>>` [blue]`isList([<>])` +
[green]`true` +
`>>>` [blue]`isList([<1, "2">])` +
[green]`true` +
`>>>` [blue]`isList(1,2)` +
[red]`Eval Error: isList(): too many params -- expected 1, got 2`

===== isNil()
Syntax: `isNil(<expr>) -> bool` +
Returns _true_ if the value type of _<expr>_ is nil, false otherwise.

.Examples
`>>>` [blue]`isNil(nil)` +
[green]`true` +
`>>>` [blue]`isNil(1)` +
[green]`false`

===== isRational()
Syntax: `isRational(<expr>) -> bool` +
Returns _true_ if the value type of _<expr>_ is fraction or int, false otherwise.

.Examples
`>>>` [blue]`isRational(4.5)` +
[green]`false` +
`>>>` [blue]`isRational(4:5)` +
[green]`true` +
`>>>` [blue]`isRational(4)` +
[green]**`true`** +
`>>>` [blue]`isRational(1.(3))` +
[green]`true`

===== isString()
Syntax: `isString(<expr>) -> bool` +
Returns _true_ if the value type of _<expr>_ is string, false otherwise.

.Examples
`>>>` [blue]`isString("ciao")` +
[green]`true` +
`>>>` [blue]`isString(2)` +
[green]`false` +
`>>>` [blue]`isString(2+"2")` +
[green]`true`

===== bool()
Syntax: `bool(<expr>) -> bool` +
Returns a _boolean_ value consistent with the value of the expression.

.Examples
`>>>` [blue]`bool(1)` +
[green]`true` +
`>>>` [blue]`bool(0)` +
[green]`false` +
`>>>` [blue]`bool("")` +
[green]`false` +
`>>>` [blue]`bool([])` +
[green]`false` +
`>>>` [blue]`bool([1])` +
[green]`true` +
`>>>` [blue]`bool({})` +
[green]`false` +
`>>>` [blue]`bool({1: "one"})` +
[green]`true`

===== int()
Syntax: `int(<expr>) -> int` +
Returns an _integer_ value consistent with the value of the expression.

.Examples
`>>>` [blue]`int(2)` +
[green]`2` +
`>>>` [blue]`int("2")` +
[green]`2` +
`>>>` [blue]`int("0x1")` +
[red]`Eval Error: strconv.Atoi: parsing "0x1": invalid syntax` +
`>>>` [blue]`int(0b10)` +
[green]`2` +
`>>>` [blue]`int(0o2)` +
[green]`2` +
`>>>` [blue]`int(0x2)` +
[green]`2` +
`>>>` [blue]`int(1.8)` +
[green]`1` +
`>>>` [blue]`int(5:2)` +
[green]`2` +
`>>>` [blue]`int([])` +
[red]`Eval Error: int(): can't convert list to int` +
`>>>` [blue]`int(true)` +
[green]`1` +
`>>>` [blue]`int(false)` +
[green]`0`


===== float()
Syntax: `float(<expr>) -> float` +
Returns a _float_ value consistent with the value of the expression.

.Examples
`>>>` [blue]`float(2)` +
[green]`2` +
`>>>` [blue]`float(2.1)` +
[green]`2.1` +
`>>>` [blue]`float(2.3(1))` +
[green]`2.311111111111111` +
`>>>` [blue]`float("3.14")` +
[green]`3.14` +
`>>>` [blue]`float(3:4)` +
[green]`0.75` +
`>>>` [blue]`float(true)` +
[green]`1` +
`>>>` [blue]`float(false)` +
[green]`0`

===== string()
Syntax: `string(<expr>) -> string` +
Returns a _string_ value consistent with the value of the expression.

.Examples
`>>>` [blue]`string(2)` +
[green]`"2"` +
`>>>` [blue]`string(0.8)` +
[green]`"0.8"` +
`>>>` [blue]`string([1,2])` +
[green]`"[1, 2]"` +
`>>>` [blue]`string({1: "one", 2: "two"})` +
[green]`"{1: "one", 2: "two"}"` +
`>>>` [blue]`string(2:5)` +
[green]`"2:5"` +
`>>>` [blue]`string(3==2)` +
[green]`"false"`

===== fract()
Syntax: `fract(<expr>) -> fraction` +
Returns a _fraction_ value consistent with the value of the expression.

.Examples
`>>>` [blue]`fract(2)` +
[green]`2:1` +
`>>>` [blue]`fract(2.5)` +
[green]`5:2` +
`>>>` [blue]`fract("2.5")` +
[green]`5:2` +
`>>>` [blue]`fract(1.(3))` +
[green]`4:3` +
`>>>` [blue]`fract([2])` +
[red]`Eval Error: fract(): can't convert array to float` +
`>>>` [blue]`fract(false)` +
[green]`0:1` +
`>>>` [blue]`fract(true)` +
[green]`1:1`

===== char()
Syntax: `char(<intexpr>) -> string` +
Returns the character whose ASCII (soon Unicode too) code point is specified by the integer expression.

.Examples
`>>>` [blue]`char(65)` +
[green]`"A"` +
`>>>` [blue]`char(97)` +
[green]`"a"`

===== eval()
Syntax: `eval(<string-expr>) -> any` +
Computes and returns the value of the [.underline]#string# expression.

.Examples
`>>>` [blue]`eval( "2 + fract(1.(3))" )` +
[green]`10:3`

===== var()
Syntax: +
 `{4sp}var(<string-expr>, <expr>) -> any` +
 `{4sp}var(<string-expr>) -> any`

This function allows you to define variables whose names can include special characters. The first form of the function allows you to define a variable with a name specified by the first parameter and assign it the value of the second parameter. The second form only returns the value of the variable with the specified name.

.Examples
`>>>` [blue]`var("$x", 3+9)` +
[green]`12` +
`>>>` [blue]`var("$"+"x")` +
[green]`12` +
`>>>` [blue]`var("gain%", var("$x"))` +
[green]`12` +
`>>>` [blue]`var("gain%", var("$x")+1)` +
[green]`13`

===== set
Syntax: +
 `{4sp}set(<string-expr>, <expr>) -> any`

This function allows you to set the value of a variable whose name can include special characters. The first parameter is the name of the variable and the second parameter is the new value to assign to that variable.

It is equivalent to the first form of the var() function, but it is more explicit about the intent of changing the value of an existing variable. 

.Examples
`>>>` [blue]`set("$x", 100)` +
[green]`100` +
`>>>` [blue]`var("$x")` +
[green]`100` +


==== Module "fmt"
#to-do#

===== print()

===== println()

==== Module "import"
Module activation: +
`{4sp}BUILTIN "import"`

===== _import()_
Syntax: +
`{4sp}import(<source-file>)`

Loads the multi-expression contained in the specified source and returns its value.

===== _importAll()_

==== Module "iterator"
Module activation: +
`{4sp}BUILTIN "iterator"`

===== run()
Syntax: +
`{4sp}run(<iterator>, <operator>, <vars>) -> any`

where:
[horizontal]
iterator:: is any iterator.
operator:: is a function that accepts two optional parameters: _index_ and _item_. 
vars:: is a dictionary defining a set of variables that are used to initialize the local context of run() and that will be inherited by each call to the operator.

TIP: Instead of using _index_ and _item_ parameter, the operator function can refere the two automatic variables `$_` and `$__` respectively.

_run()_ iterates over the specified iterator and applies the specified operator to the current value of the iterator.
It returns the value of the variable _status_ it defined, otherwise _nil_.

In addition to the variables defined in vars, the local context includes the function _abort()_. The operator can call _abort()_ to terminate iterations.

_abort()_ accepts the optional parameter _status_ used to change the value of of the variable of the same name and, therefore, of the value returned by _run()_.

.Examples
`>>>` [blue]`builtin "iterator"` [gray]__// enable the iterator builtin module__ +
[green]`1` +
`>>>` [blue]`run($(10..15), func(){status = $\_})` [gray]__// returns the last iteration's index__ +
[green]`4` +
`>>>` [blue]`run($(10..15), func(){status = $__})` [gray]_// returns the last iteration's item_ +
[green]`14` +
`>>>` [blue]`run($(1..5), func(index,item){ (index > 3) ? {abort()} :: {status = item} })` +
[green]`4` +
`>>>` [blue]`run($(1..5), func(index,item){ (index > 2) ? {abort("aborted")} :: {status = item} })` +
[green]`aborted` 


==== Module "math.arith"
Module activation: +
`{4sp}BUILTIN "math.arith"`

Currently, the "math.arith" module provides two functions, _add()_ and _mul()_, that perform addition and multiplication of an arbitrary number of parameters. More functions will be added in the future.

* <<_add,add()>>
* <<_mul,mul()>>


===== add()
Syntax: +
`{4sp}add(<num-expr1>, <num-expr2>, ...) -> any` +
`{4sp}add(<list-of-num-expr>]) -> any` +
`{4sp}add(<iterator-over-num-values>) -> any`

Returns the sum of the values of the parameters. The parameters can be of any numeric type for which the [blue]`+` operator is defined. The result type depends on the types of the parameters. If all parameters are of the same type, the result is of that type. If the parameters are of different types, the result is of the type that can represent all the parameter types without loss of information. For example, if the parameters are a mix of integers and floats, the result is a float. If the parameters are a mix of number types, the result has the type of the most general one.

.Examples
`>>>` [blue]`builtin "math.arith"` +
[green]`1` +
`>>>` [blue]`add(1,2,3)` +
[green]`6` +
`>>>` [blue]`add(1.1,2.1,3.1)` +
[green]`6.300000000000001` +
`>>>` [blue]`add("1","2","3")` +
[red]`Eval Error: add(): param nr 1 (1 in 0) has wrong type string, number expected` +
`>>>` [blue]`add(1:3, 2:3, 3:3)` +
[green]`2:1` +
`>>>` [blue]`add(1, "2")` +
[red]`Eval Error: add(): param nr 2 (2 in 0) has wrong type string, number expected` +
`>>>` [blue]`add([1,2,3])` +
[green]`6` +
`>>>` [blue]`iterator=$([1,2,3]); add(iterator)` +
[green]`6` +
`>>>` [blue]`add($([1,2,3]))` +
[green]`6`

===== mul()
Syntax: +
`{4sp}mul(<num-expr1>, <num-expr2>, ...) -> any` +
`{4sp}mul(<list-of-num-expr>]) -> any` +
`{4sp}mul(<iterator-over-num-values>) -> any`

Same as <<_add,add()>> but returns the product of the values of the parameters.

.Examples
`>>>` [blue]`builtin "math.arith"` +
[green]`1` +
`>>>` [blue]`mul(2,3,4)` +
[green]`24`

==== Module "os.file"
Module activation: +
`{4sp}BUILTIN "os.file"`

The "os.file" module provides functions for working with files.

.File related functions
* <<_fileOpen,fileOpen()>>
* <<_fileAppend,fileAppend()>>
* <<_fileCreate,fileCreate()>>
* <<_fileClose,fileClose()>>
* <<_fileWriteText,fileWriteText()>>
* <<_fileReadText,fileReadText()>>
* <<_fileReadTextAll,fileReadTextAll()>>

.Iterator functions for files
* <<_fileByteIterator,fileByteIterator()>>
* <<_fileLineIterator,fileLineIterator()>>

More functions will be added in the future.

---

===== fileOpen()
Syntax: +
`{4sp}fileOpen(<file-path>) -> file-handle`

Returns a file handle for the specified file path. The file is opened in read-write mode. If the file does not exist, it is created.

===== fileAppend()
Syntax: +
`{4sp}fileAppend(<file-path>) -> any`

Like <<_fileCreate,fileCreate()>> but write operations happen at the end of the file.

===== fileCreate()
Syntax: +
`{4sp}fileCreate(<file-path>) -> file-handle`

Creates or truncates the named _<file-path>_. If the file already exists, it is truncated. If the file does not exist, it is created with mode 0o666 (before umask). The associated file descriptor has mode _O_RDWR_. The directory containing the file must already exist.

===== fileClose()
#to-do#

===== fileWriteText()
#to-do#

===== fileReadText()
#to-do#

===== fileReadTextAll()
#to-do#

===== fileByteIterator()
Syntax: +
`{4sp}fileByteIterator(handle-or-path) -> iterator`

Returns an iterator that produces the bytes of the specified file. The parameter can be either a file handle or a file path. If a file path is provided, the file is opened and closed automatically by the iterator.

.Examples
`>>>` [blue]`builtin "os.file"` +
[green]`1` +
`>>>` [blue]`it=fileByteIterator("test-file.txt") map $_` +
[green]`$($(fileByteIterator@"test-file.txt"))` +
`>>>` [blue]`$$( it )` +
[green]`[<
  117,
  110,
  111,
  10,
  100,
  117,
  101,
  10
>]`

`>>>` [blue]`builtin ["os.file", "string"]` +
[green]`2` +
`>>>` [blue]`it = fileByteIterator("test-file.txt") map char($_)` +
[green]`$($(fileByteIterator@"test-file.txt"))` +
`>>>` [blue]`$$(it))` +
[green]`[<
  "u",
  "n",
  "o",
  "
",
  "d",
  "u",
  "e",
  "
",
>]`

===== fileLineIterator()
Syntax: +
`{4sp}fileLineIterator(handle-or-path) -> iterator`

Returns an iterator that produces the lines of the specified file. The parameter can be either a file handle or a file path. If a file path is provided, the file is opened and closed automatically by the iterator.

.Examples
`>>>` [blue]`builtin "os.file"` +
[green]`1` +
`>>>` [blue]`it = fileLineIterator("test-file.txt") map $_` +
[green]`$($(fileLineIterator@"test-file.txt"))` +
`>>>` [blue]`$$(it)` +
[green]`[
  "uno",
  "due"
]`


==== Module "string"
Module activation: +
`{4sp}BUILTIN "string"`

This module provides functions for working with strings.

Currently available functions:

* <<_strJoin,strJoin()>>
* <<_strSub,strSub()>>
* <<_strSplit,strSplit()>>
* <<_strTrim,strTrim()>>
* <<_strStartsWith,strStartsWith()>>
* <<_strEndsWith,strEndsWith()>>
* <<_strUpper,strUpper()>>
* <<_strLower,strLower()>>

---

===== strJoin()
Syntax: +
`{4sp}strJoin(<separator>, <item> ...) -> string`

Returns a string obtained by concatenating the string items (starting from the second argument), separated by the string value of the separator.

.Examples
`>>>` [blue]`strJoin(", ", "one", "two", "three")` +
[green]`"one, two, three"` +
`>>>` [blue]`strJoin(", ", ["one", "two", "three"])` +
[green]`"one, two, three"`

===== strSub()
Syntax: +
`{4sp}strSub(<string>, <start-index>=0, <count>=-1) -> string`

Returns a substring of the specified string starting at the specified index and having the specified length. If the start index is negative, it is interpreted as an offset from the end of the string. If the count is negative, it means to take all characters until the end of the string.

.Examples
`>>>` [blue]`strSub("Hello, world!", 7, 5)` +
[green]`"world"` +
`>>>` [blue]`strSub("Hello, world!", -6, 5)` +
[green]`"world"` +
`>>>` [blue]`strSub("Hello, world!", 7)` +
[green]`"world!"` +
`>>>` [blue]`strSub("Hello, world!", -6)` +
[green]`"world!"`

===== strSplit()
Syntax: +
`{4sp}strSplit(<string>, <separator>="", <count>=-1) -> list`

Returns a list of substrings obtained by splitting the specified string using the specified separator. If the separator is an empty string, the string is split into individual characters. If the count is negative, it means to split all occurrences of the separator.

.Examples
`>>>` [blue]`strSplit("one, two, three", ", ")` +
[green]`["one", "two", "three"]` +
`>>>` [blue]`strSplit("one, two, three", ", ", 2)` +
[green]`["one", "two, three"]` +
`>>>` [blue]`strSplit("one, two, three")` +
[green]`["o", "n", "e", ",", " ", "t", "w", "o", ",", " ", "t", "h", "r", "e", "e"]`

===== strTrim()
Syntax: +
`{4sp}strTrim(<string>) -> string`

Returns a string obtained by removing leading and trailing whitespace characters from the specified string.

.Examples
`>>>` [blue]`strTrim("  Hello, world!  ")` +
[green]`"Hello, world!"`

===== strStartsWith()
Syntax: +
`{4sp}strStartsWith(<string>, <prefix> ...) -> bool`

Returns a boolean indicating whether the specified string starts with any of the given prefixes.

.Examples
`>>>` [blue]`strStartsWith("Hello, world!", "Hello")` +
[green]`true` +
`>>>` [blue]`strStartsWith("Hello, world!", "world")` +
[green]`false` +
`>>>` [blue]`strStartsWith("Hello, world!", "Hi", "He")` +
[green]`true`


===== strEndsWith()
Syntax: +
`{4sp}strEndsWith(<string>, <suffix> ...) -> bool`

Returns a boolean indicating whether the specified string ends with any of the given suffixes.

.Examples
`>>>` [blue]`strEndsWith("Hello, world!", "world!")` +
[green]`true` +
`>>>` [blue]`strEndsWith("Hello, world!", "Hello")` +
[green]`false` +
`>>>` [blue]`strEndsWith("Hello, world!", "Hi", "world!")` +
[green]`true`


===== strUpper()
Syntax: +
`{4sp}strUpper(<string>) -> string`

Returns a string obtained by converting all characters of the specified string to uppercase.

.Examples
`>>>` [blue]`strUpper("Hello, world!")` +
[green]`"HELLO, WORLD!"`

===== strLower()
Syntax: +
`{4sp}strLower(<string>) -> string`

Returns a string obtained by converting all characters of the specified string to lowercase.

.Examples
`>>>` [blue]`strLower("Hello, world!")` +
[green]`"hello, world!"`



== Iterators
Iterators are objects that can be used to traverse collections, such as lists and dictionaries. They are created by providing a _data-source_ object, the collection, in a `$(<data-source>)` expression. Once an iterator is created, it can be used to access the elements of the collection one by one.

In general, data-sources are objects that can be iterated over. They are defined as dictionaries having the key `next` whose value is a function that returns the next element of the collection and updates the state of the iterator. The _next_ function must return the special value [blue]_nil_ when there are no more elements to iterate over.

Lists and dictionaries are implicit data-sources. The syntax for creating an iterator is as follows.

.Iterator creation syntax
====
*_iterator_* = "**$(**" _data-source_ "**)**" +
_data-source_ = _explicit_ | _array_spec_ | _linked-list-spec_ | _dict-spec_ | _custom-data-source_

_explicit_ = _any-expr_ { "**,**" _any-expr_ }

_array-spec_ = _array_  ["**,**" _range-options_] +
_array_ = "**[**" _any-expr_ { "**,**" _any-expr_ } "**]**" +
_range-options_ = _start-index_ [ "**,**" _end-index_ [ "**,**" _step_ ]] +
_start-index_, _end-index_, _step_ = _integer-expr_

_dict-spec_ = _dict_  ["**,**" _dict-options_] +
_dict_ = "**{**" _key-value-pair_ { "**,**" _key-value-pair_ } "**}**" +
_key-value-pair_ = _scalar-value_ "**:**" _any-expr_ +
_scalar-value_ = _string_ | _number_ | _boolean_ +
_dict-options_ = _sort-order_ [ "**,**" _iter-mode_ ] +
_sort-order_ = _asc-order_ | _desc-order_ | _no-sort_ | _default-order_ +
_asc-order_ = "**asc**" | "**a**" +
_desc-order_ = "**desc**" | "**d**" +
_no-sort_ = "**none**" | "**nosort**" | "**no-sort**" | "**n**" +
_default-order_ = "**default**" | "" +
_iter-mode_ = _keys-iter_ | _values-iter_ | _items-iter_ | _default-iter_ +
_keys-iter_ = "**key**" | "**keys**" | "**k**" +
_values-iter_ = "**value**" | "**values**" | "**v**" +
_items-iter_ = "**item**" | "**items**" | "**i**" +
_default-iter_ = "**default**" | ""

_custom-data-source_ = _dict_ having the key `next` whose value is a function that returns the next element of the collection and updates the state of the iterator.
====

NOTE: Currently, _default-order_ is the same as _asc-order_. In the future, it will be possible to specify a custom sorting function to define the default order.

NOTE: Currently, _default-iter_ is the same as _keys-iter_. In the future, it will be possible to specify a custom iteration function to define the default iteration mode.

.Example: iterator over an explicit list of elements
`>>>` [blue]`it = $("A", "B", "C")` +
[green]`$(#3)` +
`>>>` [blue]`it{plusplus}` +
[green]`"A"` +
`>>>` [blue]`it{plusplus}` +
[green]`"B"` +
`>>>` [blue]`it{plusplus}` +
[green]`"C"` +
`>>>` [blue]`it{plusplus}` +
[red]`Eval Error: EOF`

.Example: iterator over an array
`>>>` [blue]`it = $(["one", "two", "three"])` +
[green]`$([#3])` +
`>>>` [blue]`it{plusplus}` +
[green]`"one"` +
`>>>` [blue]`it{plusplus}` +
[green]`"two"` +
`>>>` [blue]`it{plusplus}` +
[green]`"three"` +
`>>>` [blue]`it{plusplus}` +
[red]`Eval Error: EOF`

When creating an array-type iterator expression, you can specify a range of indices and a step to iterate over a subset of the array.

ifdef::show_todo[]
#TODO: implement same functionality for linked lists and dictionaries#
endif::[]

Indexing starts at 0. If the start index is not specified, it defaults to 0. If the end index is not specified, it defaults to the index of the last element of the array. If the step is not specified, it defaults to 1.

Negative indexes are allowed. They are interpreted as offsets from the end of the array. For example, an end index of -1 means the index of the last element of the array, an end index of -2 means the index of the second to last element of the array, and so on.

Negative steps are also allowed. They are interpreted as reverse iteration. For example, a step of -1 means to iterate over the array in reverse order.

.Example: iterator over an array with index, range and step 
`>>>` [blue]`it = $([1, 2, 3, 4, 5], 1, 4, 2)` +
[green]`$([#5])` +
`>>>` [blue]`it{plusplus}` +
[green]`2` +
`>>>` [blue]`it{plusplus}` +
[green]`4` +
`>>>` [blue]`it{plusplus}` +
[red]`Eval Error: EOF`

.Example: iterator over an array in reverse order
`>>>` [blue]`it = $([1, 2, 3], -1, 0, -1)` + 
[green]`$([#3])` +
`>>>` [blue]`it{plusplus}` +
[green]`3` +
`>>>` [blue]`it{plusplus}` +
[green]`2` +
`>>>` [blue]`it{plusplus}` +
[green]`1` +
`>>>` [blue]`it{plusplus}` +
[red]`Eval Error: EOF`


=== Operators on iterators
The above example shows the use of the [blue]`{plusplus}` operator to get the next element of an iterator. The [blue]`{plusplus}` operator is a postfix operator that can be used with iterators. It returns the next element of the collection and updates the state of the iterator. When there are no more elements to iterate over, it returns the error [red]_Eval Error: EOF_.


After the first use of the [blue]`{plusplus}` operator, the prefixed operator [blue]`{star}` operator can be used to get the current element of the collection without updating the state of the iterator. When there are no more elements to iterate over, it returns the error [red]_Eval Error: EOF_. Same error is returned if the [blue]`{star}` operator is used before the first use of the [blue]`{plusplus}` operator.

.Example: using the [blue]`{star}` operator
`>>>` [blue]`it = $(["one", "two", "three"])` +
[green]`$([#3])` +
`>>>` [blue]`{star}it` +
[red]`Eval Error: EOF` +
`>>>` [blue]`it{plusplus}` +
[green]`"one"` +
`>>>` [blue]`{star}it` +
[green]`"one"`

==== [blue]*`$$()`* -- Expansion special function for iterators
The [blue]`$$()` operator is a special function already seen applied to contexts. It can also be applied to iterators. When applied to an iterator, it visits all remaining elements of the iterator and returns them as a _linked list_. The state of the iterator is updated to the end of the collection. If there are no more elements to iterate over, it returns an empty list.

#todo: examples#

==== Named operators
Named operators are operators that are identified by a name instead of a symbol. They are implicitly defined and can be called using a special syntax. For example, the [blue]`{plusplus}` has the equivalent named operator [blue]`.next`.

.Available named operators
* *_.next_*: same as [blue]`{plusplus}`.
* *_.current_*: same as [blue]`{star}.`
* *_.reset_*: resets the state of the iterator to the initial state.
* *_.count_*: returns the number of elements in the iterator already visited.
* *_.index_*: returns the index of the current element in the iterator. Before the first use of the [blue]`{plusplus}` operator, it returns the invalid value [red]_-1_.

TIP: Iterators built on custom data-sources can provide additional named operators, depending on the functionality they want to expose. For example, an iterator over a list could provide a named operator called [blue]`.reverse` that returns the next element of the collection in reverse order.

.Example: using the named operators
`>>>` [blue]`it = $(["one", "two", "three"])` +
[green]`$([#3])` +
`>>>` [blue]`it.next` +
[green]`"one"` +
`>>>` [blue]`it.current` +
[green]`"one"` +
`>>>` [blue]`it.next` +
[green]`"two"` +
`>>>` [blue]`it.reset` +
[green]`<nil>` +
`>>>` [blue]`it.current` +
[red]`Eval Error: EOF` +
`>>>` [blue]`it.next` +
[green]`"one"`

=== Infixed operators on iterators
There are also some infixed operators that can be used with iterators. They are defined as follows.

* <<_cat,cat operator>>
* <<_digest,digest operator>>
* <<_filter,filter operator>>
* <<_groupby,groupby operator>>
* <<_map,map operator>>
//* <<_reduce,reduce operator>>: applies a binary expression cumulatively to the elements of the iterator, from left to right, to reduce the iterator to a single value.
//* <<_zip,zip operator>>: takes two or more iterators and returns a list of tuples, where the i-th tuple contains the i-th element from each of the input iterators.


==== Automatic variables in operators
At each iteration, the following automatic variables are available for use in the expression of the [blue]`digest`, [blue]`filter`, [blue]`groupby`, and [blue]`map` operators.

* `$_`: the current element of the iterator.
* `$__`: the index of the current element in the iterator, starting from 0.
* `$#`: the number of elements already visited in the iterator, starting from 0.

---

==== [blue]`cat` operator
Syntax: +
`{4sp}<iterable> cat <iterable> -> <iterator>{4sp}`

[blue]`cat` operator takes two iterators or iterables and returns a new iterator that produces the elements of the first iterator followed by the elements of the second iterator.

.Examples
`>>>` [blue]`it = [1,2] cat [3] cat [4,5]` +
[green]`$($($([#2]) cat $([#1])) cat $([#2]))` +
`>>>` [blue]`$$(it)` +
[green]`[<1, 2, 3, 4, 5>]`


==== [blue]`digest` operator
Syntax: +
`{4sp}<iterable> digest <expr> -> <any>{4sp}`

[blue]`digest` operator takes one iterator or iterable and an expression. It evaluates the expression for each element of the iterator and returns the last computed value.

.Examples
`>>>` [blue]`it = $(2,5)` +
[green]`$(2..5..1)` +
`>>>` [blue]`sum=0; it digest (sum+=$_)` +
[green]`9`

.Some notes on the expression used with the [blue]`digest` operator
[NOTE]
====
* All assign operators have a lower precedence than the [blue]`digest` operator. Therefore, parentheses are required around the expression to ensure that the assignment is evaluated before the [blue]`digest` operator.
* Since the [blue]`digest` operator evaluates the expression for each element of the iterator, it can be used to perform side effects, such as updating a variable.
* At each iteration, the automatic variables `$_`, `$__`, `$#`, and `last` are available for use in the expression. In the above example, the _sum_ variable can be replaced by the automatic variable _last_ to get the same result. The _last_ variable is initialized to 0 before the first iteration and is updated to the last computed value after each iteration. Therefore, the above example can be rewritten as follows.

`>>>` [blue]`it = $(2,5)` +
[green]`$(2..5..1)` +
`>>>` [blue]`last=0; it digest last+$_` +
[green]`9`
====

==== [blue]`filter` operator
Syntax: +
`{4sp}<iterable> filter <expr> -> <iterator>{4sp}`

[blue]`filter` applies a boolean expression to each element of the iterator and returns a new iterator that only produces the elements of the first iterator for which the expression evaluates to true.

.Examples
`>>>` [blue]`it = $(["one", "two", "three"])` +
[green]`$([#3])` +
`>>>` [blue]`filtered_it = it filter $_ == "two"` +
[green]`$($([#1]))` +
`>>>` [blue]`$$(filtered_it)` +
[green]`[<"two">]`

==== [blue]`groupby` operator
Syntax: +
`{4sp}<list-of-dicts> groupby <key> -> <dict>{4sp}`

The left side of [blue]`groupby` operator is a list of dictionaries or an iterator over a list of dictionaries. It takes a key and returns a dictionary where the keys are the unique values of the specified key in the dictionaries and the values are lists of dictionaries that have that key value. In other words, it groups the dictionaries by the specified key value.

NOTE: Currently, keys of group are always strings. In the future, it will be possible to specify a key function to compute the keys of the groups.

.Examples
`>>>` [blue]`[{"name": "Alice", "age": 30}, {"name": "Bob", "age": 25}, {"name": "Charlie", "age": 30}] groupby "age"` +
[source,yaml]
----
{
  "25": [
    {
      "name": "Bob",
      "age": 25
    }
  ],
  "30": [
    {
      "name": "Alice",
      "age": 30
    },
    {
      "name": "Charlie",
      "age": 30
    }
  ]
}
----

==== [blue]`map` operator
Syntax: +
`{4sp}<iterable> map <expr> -> <iterator>{4sp}`

[blue]`map` operator iterates over the elements of the iterator and evaluates the expressions provided on the right side for each element. Its result is a new iterator over the computed values of the that expression. The current element of the iterator is available in the expression as the variable `$_`.

.Example: using the [blue]`map` operator
`>>>` [blue]`it = $(["one", "two", "three"])` +
[green]`$([#3])` +
`>>>` [blue]`excl_it = it map $_ + "!"` +
[green]`$($([#3]))` +
`>>>` [blue]`$$(excl_it)` +
[green]`[<"one!", "two!", "three!">]`

=== Iterator over custom data-sources
It is possible to create iterators over custom data-sources by defining a dictionary that has the key `next` whose value is a function that returns the next element of the collection and updates the state of the iterator. The syntax for creating an iterator over a custom data-source is as follows.

#TODO: custom data-sources#



== Plugins
#TODO: plugins#


