Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions app/rust-ffi/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,7 @@ pub fn is_numeric_literal(code: &str) -> bool {
enso_parser::syntax::tree::Variant::Number(_) => true,
enso_parser::syntax::tree::Variant::UnaryOprApp(app) =>
app.opr.code == "-"
&& app.rhs.as_ref().map_or(false, |rhs| {
matches!(rhs.variant, enso_parser::syntax::tree::Variant::Number(_))
}),
&& matches!(&app.rhs.variant, enso_parser::syntax::tree::Variant::Number(_)),
_ => false,
}
}
Expand Down
7 changes: 3 additions & 4 deletions app/ydoc-shared/src/ast/parse.ts
Original file line number Diff line number Diff line change
Expand Up @@ -240,10 +240,9 @@ class Abstractor {
node = Wildcard.concrete(this.module, token)
break
}
// These expression types are (or will be) used for backend analysis.
// The frontend can ignore them, avoiding some problems with expressions sharing spans
// (which makes it impossible to give them unique IDs in the current IdMap format).
case RawAst.Tree.Type.OprSectionBoundary:
// This expression type is not yet consistent with the backend's semantics.
// The frontend can ignore it, avoiding some problems with expressions sharing spans
// (which makes it impossible to give assign unique IDs in the current IdMap format).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is now about TemplateFunction, right?

case RawAst.Tree.Type.TemplateFunction:
return { whitespace, node: this.abstractExpression(tree.ast).node }
case RawAst.Tree.Type.Invalid: {
Expand Down
29 changes: 20 additions & 9 deletions docs/syntax/functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -417,15 +417,26 @@ result = 1..100 . each random . take 10 . sort
### Sections

An operator section is a nice shorthand for partially applying an operator. It
works as follows.

- Where an argument is not applied to an operator, the missing argument is
replaced by an implicit `_`.
- The application is then translated based upon the rules for
[underscore parameters](./function-parameters.md#underscore-parameters)
described later.
- The whitespace-based precedence rules discussed above also apply to operator
sections.
works as follows:

- When a binary operator is referred to without providing parameters, its value
is a function with two parameters (left followed by right). This can be
written either by wrapping the operator in parentheses, or by using it in an
unspaced expression without any operand:
```enso
[1, 2, 3].reduce (+) . should_equal 6
[1, 2, 3].reduce function=+ . should_equal 6
```
- When a binary operator is applied to a single parameter, the result is a
function accepting the parameter.
```enso
[1, 2, 3].map (1 /) . should_equal [1.0, 0.5, 0.3333333333333333]
[1, 2, 3].map (/ 1) . should_equal [1.0, 2.0, 3.0]
[1, 2, 3].map function=1/ . should_equal [1.0, 0.5, 0.3333333333333333]
[1, 2, 3].map function=/1 . should_equal [1.0, 2.0, 3.0]
[1, 2, 3].map 1/ . should_equal [1.0, 0.5, 0.3333333333333333]
[1, 2, 3].map /1 . should_equal [1.0, 2.0, 3.0]
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for modifying the documentation!


## Mixfix Functions

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1153,7 +1153,6 @@ yield switch (translateExpression(group.getBody(), false)) {
.build();
}
case Tree.Function fun -> translateFunction(fun);
case Tree.OprSectionBoundary bound -> translateExpression(bound.getAst(), false);
case Tree.UnaryOprApp un when "-".equals(un.getOpr().codeRepr()) ->
switch (translateExpression(un.getRhs(), false)) {
case Literal.Number n ->
Expand All @@ -1180,8 +1179,6 @@ yield switch (translateExpression(group.getBody(), false)) {
.location(getIdentifiedLocation(un))
.build();
}
case null ->
translateSyntaxError(tree, new Syntax.UnsupportedSyntax("Strange unary -"));
};
case Tree.TemplateFunction templ -> translateExpression(templ.getAst(), false);
case Tree.Wildcard wild -> new Name.Blank(getIdentifiedLocation(wild), meta());
Expand Down Expand Up @@ -1400,7 +1397,6 @@ Tree applySkip(Tree tree) {
yield tree;
}
case Tree.UnaryOprApp app -> app.getRhs();
case Tree.OprSectionBoundary section -> section.getAst();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, looks like a simplification.

case Tree.TemplateFunction function -> function.getAst();
case Tree.AnnotatedBuiltin annotated -> annotated.getExpression();
case Tree.ExpressionStatement statement -> statement.getExpression();
Expand Down
116 changes: 59 additions & 57 deletions lib/rust/parser/debug/tests/parse.rs

Large diffs are not rendered by default.

32 changes: 15 additions & 17 deletions lib/rust/parser/src/syntax/expression/apply.rs
Original file line number Diff line number Diff line change
Expand Up @@ -61,44 +61,42 @@ impl<'s> ApplyOperator<'s> {
.first()
.and_then(|token| token.operator_properties().unwrap().lhs_section_termination())
{
// Section-terminating special operators

let lhs = match lhs_termination {
SectionTermination::Reify => lhs.map(Tree::from),
// This mainly serves to handle blanks in the LHS of an ("old-style") lambda,
// preventing them from being treated as creating a template function: In that case,
// the argument definition is parsed before the operator is encountered, so we don't
// know to treat it as a pattern until we have already finished parsing it.
SectionTermination::Unwrap => lhs.map(|op| op.value),
};
let rhs = rhs_.map(Tree::from);
let ast = syntax::tree::apply_operator(lhs, tokens, rhs);
MaybeSection::from(ast)
MaybeSection::from(syntax::tree::apply_operator(lhs, tokens, rhs))
} else if tokens.len() < 2
&& tokens.first().is_some_and(|opr| !opr.is_syntactic_binary_operator())
{
// Normal, section-forming operator

let mut rhs = None;
let mut elided = 0;
let mut wildcards = 0;
if let Some(rhs_) = rhs_ {
if reify_rhs_section {
rhs = Some(Tree::from(rhs_));
} else {
rhs = Some(rhs_.value);
elided += rhs_.elided;
wildcards += rhs_.wildcards;
}
}
elided += lhs.is_none() as u32 + rhs.is_none() as u32;
let mut operand =
MaybeSection::from(lhs).map(|lhs| syntax::tree::apply_operator(lhs, tokens, rhs));
operand.elided += elided;
operand.wildcards += wildcards;
operand
} else {
// Non-section-forming (but not section-terminating) operator, or multiple-operator
// error.

let rhs = rhs_.map(Tree::from);
let mut elided = 0;
if tokens.len() != 1 || !tokens[0].is_syntactic_binary_operator() {
elided += lhs.is_none() as u32 + rhs.is_none() as u32;
}
let mut operand =
MaybeSection::from(lhs).map(|lhs| syntax::tree::apply_operator(lhs, tokens, rhs));
operand.elided += elided;
operand
MaybeSection::from(lhs).map(|lhs| syntax::tree::apply_operator(lhs, tokens, rhs))
};
if let Some(warnings) = warnings {
warnings.apply(&mut operand.value);
Expand Down Expand Up @@ -137,9 +135,9 @@ impl<'s> ApplyUnaryOperator<'s> {

pub fn finish(self) -> MaybeSection<Tree<'s>> {
let Self { token, rhs, error, warnings } = self;
MaybeSection::new(rhs).map(|rhs| {
MaybeSection::from(rhs).map(|rhs| {
let mut tree = match rhs {
Some(rhs) => Tree::unary_opr_app(token, Some(rhs)),
Some(rhs) => Tree::unary_opr_app(token, rhs),
None =>
Tree::opr_app(None, Ok(token.with_variant(token::variant::Operator())), None)
.with_error(SyntaxError::SyntacticOperatorMissingOperandUnary),
Expand Down
2 changes: 2 additions & 0 deletions lib/rust/parser/src/syntax/expression/arity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,8 @@ where Inner: NamedOperandConsumer<'s> + OperatorConsumer<'s>
(Some(_), Some(_)) => None,
};
let properties = token.operator_properties().unwrap();
// The spacing logic here only affects some error representations, except in the `._` case,
// which is not implemented and will probably be made a syntax error.
let reify_rhs_section = properties.can_form_section()
&& (lhs == Some(Spacing::Spaced) || rhs == Some(Spacing::Spaced));
let is_value_operation = missing.is_none() && properties.is_value_operation();
Expand Down
2 changes: 1 addition & 1 deletion lib/rust/parser/src/syntax/expression/numbers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ fn flush<'s, Inner: TokenConsumer<'s> + TreeConsumer<'s>>(
fn maybe_negated<'s>(minus: Option<Token<'s>>, tree: Tree<'s>) -> Tree<'s> {
match minus {
Some(minus) =>
Tree::unary_opr_app(minus.with_variant(token::variant::UnaryOperator()), Some(tree)),
Tree::unary_opr_app(minus.with_variant(token::variant::UnaryOperator()), tree),
None => tree,
}
}
Expand Down
35 changes: 7 additions & 28 deletions lib/rust/parser/src/syntax/expression/section.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,23 +9,19 @@ use crate::syntax::Tree;
// === MaybeSection ===
// ====================

/// Wraps a value, tracking the number of wildcards or elided operands within it.
/// Wraps a value, tracking the number of wildcards operands within it.
#[derive(Default, Debug, PartialEq, Eq)]
pub struct MaybeSection<T> {
pub value: T,
/// Number of elided operands in the subtree, potentially forming an *operator section*.
pub elided: u32,
/// Number of wildcards in the subtree, potentially forming a *template function*.
pub wildcards: u32,
}

/// Transpose. Note that an absent input will not be treated as an elided value; for that
/// conversion, use [`MaybeSection::new`].
/// Transpose.
impl<T> From<Option<MaybeSection<T>>> for MaybeSection<Option<T>> {
fn from(operand: Option<MaybeSection<T>>) -> Self {
match operand {
Some(MaybeSection { value, elided, wildcards }) =>
Self { value: Some(value), elided, wildcards },
Some(MaybeSection { value, wildcards }) => Self { value: Some(value), wildcards },
None => default(),
}
}
Expand All @@ -34,7 +30,6 @@ impl<T> From<Option<MaybeSection<T>>> for MaybeSection<Option<T>> {
/// Unit. Creates a MaybeSection from a node.
impl<'s> From<Tree<'s>> for MaybeSection<Tree<'s>> {
fn from(mut value: Tree<'s>) -> Self {
let elided = 0;
let wildcards = if let Tree { variant: tree::Variant::Wildcard(wildcard), .. } = &mut value
{
debug_assert_eq!(wildcard.de_bruijn_index, None);
Expand All @@ -43,42 +38,26 @@ impl<'s> From<Tree<'s>> for MaybeSection<Tree<'s>> {
} else {
0
};
Self { value, wildcards, elided }
Self { value, wildcards }
}
}

/// Counit. Bakes any information about elided operands into the tree.
impl<'s> From<MaybeSection<Tree<'s>>> for Tree<'s> {
fn from(operand: MaybeSection<Tree<'s>>) -> Self {
let MaybeSection { mut value, elided, wildcards } = operand;
if elided != 0 {
value = Tree::opr_section_boundary(elided, value);
}
let MaybeSection { mut value, wildcards } = operand;
if wildcards != 0 {
value = Tree::template_function(wildcards, value);
}
value
}
}

impl<T> MaybeSection<Option<T>> {
/// Lift an option value to a potentially-elided operand.
pub fn new(value: Option<MaybeSection<T>>) -> Self {
match value {
None => Self { value: None, elided: 1, wildcards: default() },
Some(value) => {
let MaybeSection { value, elided, wildcards } = value;
Self { value: Some(value), elided, wildcards }
}
}
}
}

impl<T> MaybeSection<T> {
/// Operate on the contained value without altering the elided-operand information.
pub fn map<U>(self, f: impl FnOnce(T) -> U) -> MaybeSection<U> {
let Self { value, elided, wildcards } = self;
let Self { value, wildcards } = self;
let value = f(value);
MaybeSection { value, elided, wildcards }
MaybeSection { value, wildcards }
}
}
6 changes: 1 addition & 5 deletions lib/rust/parser/src/syntax/expression/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -212,12 +212,8 @@ pub enum OperandMaybeNamed<'s> {

/// Operator-section/template-function termination behavior of an operator with regard to an
/// operand.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Default)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub enum SectionTermination {
/// If the operand is an operator-section/template-function, indicate it by wrapping it in a
/// suitable node.
#[default]
Reify,
/// Discard any operator-section/template-function properties associated with the operand.
Unwrap,
}
1 change: 0 additions & 1 deletion lib/rust/parser/src/syntax/expression/whitespace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,6 @@ fn tree_starts_new_no_space_group(tree: &Tree) -> bool {
| OprApp(_)
| UnaryOprApp(_)
| AutoscopedIdentifier(_)
| OprSectionBoundary(_)
| TemplateFunction(_)
| MultiSegmentApp(_)
| TypeDef(_)
Expand Down
8 changes: 4 additions & 4 deletions lib/rust/parser/src/syntax/statement.rs
Original file line number Diff line number Diff line change
Expand Up @@ -480,7 +480,6 @@ fn to_statement<'s>(
| TextLiteral(_)
| App(_)
| NamedApp(_)
| OprApp(_)
| UnaryOprApp(_)
| AutoscopedIdentifier(_)
| MultiSegmentApp(_)
Expand All @@ -489,6 +488,7 @@ fn to_statement<'s>(
| CaseOf(_)
| Array(_)
| Tuple(_) => Ok(Expression),
OprApp(app) if app.lhs.is_some() && app.rhs.is_some() => Ok(Expression),
// Expression, but since it can only occur in tail position, it never needs an
// `ExpressionStatement` node.
BodyBlock(_) => Ok(Statement),
Expand All @@ -504,9 +504,9 @@ fn to_statement<'s>(
| Annotation(_)
| Documentation(_)
| ConstructorDefinition(_) => Ok(Statement),
// These will become an error in the future.
OprSectionBoundary(_) => Ok(Expression),
TemplateFunction(_) | Lambda(_) => Ok(Expression),
// Operator sections (fully-applied operators are matched above)
OprApp(_) => Err(SyntaxError::StmtUnexpectedFunctionExpressionOprSection),
TemplateFunction(_) | Lambda(_) => Err(SyntaxError::StmtUnexpectedFunctionExpression),
// Shouldn't be possible here, but this is not currently guaranteed by the types.
ExpressionStatement(_) => Err(SyntaxError::Internal),
} {
Expand Down
1 change: 0 additions & 1 deletion lib/rust/parser/src/syntax/token/operator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,6 @@ impl HasOperatorProperties for variant::TypeAnnotationOperator {
fn operator_properties(&self) -> OperatorProperties {
OperatorProperties {
binary_infix_precedence: Some(Precedence::TypeAnnotation),
lhs_section_termination: Some(SectionTermination::Reify),
is_compile_time: true,
rhs_is_non_expression: true,
..default()
Expand Down
17 changes: 3 additions & 14 deletions lib/rust/parser/src/syntax/tree.rs
Original file line number Diff line number Diff line change
Expand Up @@ -151,33 +151,22 @@ macro_rules! with_ast_definition { ($f:ident ($($args:tt)*)) => { $f! { $($args)
pub close: Option<token::CloseSymbol<'s>>,
},
/// Application of an operator, like `a + b`. The left or right operands might be missing,
/// thus creating an operator section like `a +`, `+ b`, or simply `+`. See the
/// [`OprSectionBoundary`] variant to learn more about operator section scope.
/// thus creating an operator section like `a +`, `+ b`, or simply `+`.
OprApp {
pub lhs: Option<Tree<'s>>,
pub opr: OperatorOrError<'s>,
pub rhs: Option<Tree<'s>>,
},
/// Application of a unary operator, like `-a` or `~handler`. It is a syntax error for `rhs`
/// to be `None`.
/// Application of a unary operator, like `-a` or `~handler`, to an operand.
UnaryOprApp {
pub opr: token::UnaryOperator<'s>,
pub rhs: Option<Tree<'s>>,
pub rhs: Tree<'s>,
},
/// Application of the autoscope operator to an identifier, e.g. `..True`.
AutoscopedIdentifier {
pub opr: token::AutoscopeOperator<'s>,
pub ident: token::Ident<'s>,
},
/// Defines the point where operator sections should be expanded to lambdas. Let's consider
/// the expression `map (.sum 1)`. It should be desugared to `map (x -> x.sum 1)`, not to
/// `map ((x -> x.sum) 1)`. The expression `.sum` will be parsed as operator section
/// ([`OprApp`] with left operand missing), and the [`OprSectionBoundary`] will be placed
/// around the whole `.sum 1` expression.
OprSectionBoundary {
pub arguments: u32,
pub ast: Tree<'s>,
},
TemplateFunction {
pub arguments: u32,
pub ast: Tree<'s>,
Expand Down
3 changes: 0 additions & 3 deletions test/Table_Tests/src/In_Memory/Table_Spec.enso
Original file line number Diff line number Diff line change
Expand Up @@ -161,9 +161,6 @@ add_specs suite_builder =
r4.pretty . should_equal "Table.new [['foo', [1, 2, 3]], ['bar', [False, True, False]], ['date', [Date.new 2022 8 27, Date.new 1999 1 1, Date.new 2012 1 23]], ['time', [Time_Of_Day.new 18, Time_Of_Day.new 1 2 34, Time_Of_Day.new 12]]]"
Debug.eval r4.pretty . should_equal r4

group_builder.specify "allows setting the column type" <|
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great if the new syntax can discover errors like this!



group_builder.specify "should handle error scenarios gracefully" <|
Table.new [["X", [1,2,3]], ["Y", [4]]] . should_fail_with Illegal_Argument
Table.new [["X", [1]], ["X", [2]]] . should_fail_with Illegal_Argument
Expand Down
Loading