Skip to content

feat(flow-php/array-dot): Path value object for dot paths #2625

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
norberttech merged 1 commit into from
Sep 12, 2026
Merged

feat(flow-php/array-dot): Path value object for dot paths #2625

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 59 additions & 13 deletions documentation/components/libs/array-dot.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,14 @@ For detailed installation instructions, see the [installation page](/documentati
```php ignore
<?php

array_dot_get(array $array, string $path) : mixed;
array_dot_set(array $array, string $path, mixed $value) : mixed;
array_dot_rename(array $array, string $path, string $newName) : mixed;
array_dot_exists(array $array, string $path) : bool;
array_dot_steps(string $path) : array;
array_dot_get(array $array, Path|string $path, ?Type $type = null) : mixed;
array_dot_set(array $array, Path|string $path, mixed $value) : array;
array_dot_rename(array $array, Path|string $path, string $newName) : array;
array_dot_exists(array $array, Path|string $path) : bool;
```

`array_dot_steps(string $path)` is deprecated, use `Path::fromString($path)->steps`.

### Dot Notation - Basic Syntax

```php
Expand All @@ -47,7 +48,7 @@ $array = [
]
];

$value = array_dot_get('foo.bar.baz'); // 1000
$value = array_dot_get($array, 'foo.bar.baz'); // 1000

$array = array_dot_set([], 'foo.bar.baz', 1000); // ['foo' => ['bar' => ['baz' => 1000]]];
```
Expand All @@ -62,7 +63,7 @@ In above example `foo.bar.baz` is path which also supports integer keys. For exa

- `?` - nullsafe
- `*` - wildcard
- `?*` - nullsafe wildcar
- `?*` - nullsafe wildcard


### Dot Notation - Custom Syntax
Expand All @@ -74,6 +75,10 @@ In above example `foo.bar.baz` is path which also supports integer keys. For exa
Supported in functions:

- `array_dot_get`
- `array_dot_exists`
- `array_dot_rename` - an absent key is left as it is

`array_dot_set` writes the key without the `?`.

Dot notation is strict by default, which means that if any step of path is not present,
function will throw exception.
Expand All @@ -91,8 +96,8 @@ $array = [
]
];

$value = array_dot_get('foo.bar.nothing'); // InvalidPathException
$value = array_dot_get('foo.bar.?nothing'); // null
$value = array_dot_get($array, 'foo.bar.nothing'); // InvalidPathException
$value = array_dot_get($array, 'foo.bar.?nothing'); // null
```

Nullsafe does not need to be used with the last step of path.
Expand All @@ -108,7 +113,7 @@ $array = [
]
];

$value = array_dot_get('foo.?bar.nothing'); // null
$value = array_dot_get($array, 'foo.?bar.nothing'); // null
```

#### Wildcard Operator - *
Expand All @@ -135,14 +140,15 @@ $array = [
]
];

$value = array_dot_get('users.*.id'); // [1, 2]
$value = array_dot_get($array, 'users.*.id'); // [1, 2]
```

#### Nullsafe Wildcard Operator - ?*

Supported in functions:

- `array_dot_get`
- `array_dot_rename` - elements without the key are left as they are

Nullsafe Wildcard operator allows to access all paths in nested arrays for non symmetric
collections.
Expand All @@ -162,7 +168,7 @@ $array = [
]
];

$value = array_dot_get('users.*.name'); // ['John']
$value = array_dot_get($array, 'users.?*.name'); // ['John']
```

#### Multipath Syntax - {}
Expand Down Expand Up @@ -192,5 +198,45 @@ $array = [
]
];

$value = array_dot_get('users.*.{id,?role}'); // [[1, null], [2, 'ADMIN']]
$value = array_dot_get($array, 'users.*.{id,?role}'); // [['id' => 1, 'role' => null], ['id' => 2, 'role' => 'ADMIN']]
```

### Dot Notation - Escaping

A backslash makes the next `\`, `.`, `?`, `*`, `,`, `{` or `}` part of the key. Before any other character the
backslash stays in the key.

```php
<?php

$array = ['a.b' => ['*' => 1], '?x' => 2, 'k,l' => 3];

$value = array_dot_get($array, 'a\.b.\*'); // 1
$value = array_dot_get($array, '\?x'); // 2
$value = array_dot_get(['m' => $array], 'm.{\?x, k\,l}'); // ['?x' => 2, 'k,l' => 3]
```

### Path

Every function accepts a `Flow\ArrayDot\Path` instead of a string. Build it from steps when the keys come from data,
so they are never parsed:

```php
<?php

use Flow\ArrayDot\Path;
use Flow\ArrayDot\Step\Key;
use Flow\ArrayDot\Step\Multimatch;
use Flow\ArrayDot\Step\Wildcard;

$path = new Path([new Key('users'), new Wildcard(), new Multimatch([
new Path([new Key('id')]),
new Path([new Key('first.name', nullsafe: true)]),
])]);

$value = array_dot_get(['users' => [['id' => 1, 'first.name' => 'John'], ['id' => 2]]], $path);
// [['id' => 1, 'first.name' => 'John'], ['id' => 2, 'first.name' => null]]

$path->toString(); // 'users.*.{id,?first\.name}'
Path::fromString('users.*.{id,?first\.name}') == $path; // true
```
54 changes: 51 additions & 3 deletions documentation/upgrading.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1435,10 +1435,10 @@ Reinstall it with the new release: `pie install flow-php/flow-php-ext`.

### 99) `flow-php/etl-adapter-csv` - `withSeparator()`, `withEnclosure()` and `withEscape()` take a single byte

| Before | After |
|----------------------------------------------------------------------------------------------------------------|------------------------------------------------------|
| Before | After |
|-----------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------|
| `withSeparator('\|\|')` / `withEnclosure('\|\|')` / `withEscape('ab')` - PHP 8.3: first byte used; PHP 8.4+: `ValueError` on read | throws `Flow\ETL\Exception\InvalidArgumentException` |
| `withSeparator('')` / `withEnclosure('')` - PHP 8.3: `,` / `"` used; PHP 8.4+: `ValueError` on read | throws `Flow\ETL\Exception\InvalidArgumentException` |
| `withSeparator('')` / `withEnclosure('')` - PHP 8.3: `,` / `"` used; PHP 8.4+: `ValueError` on read | throws `Flow\ETL\Exception\InvalidArgumentException` |

### 100) `flow-php/etl` - `array_expand()` nested in an expression gives rows, and is refused outside `withEntry()`

Expand All @@ -1449,6 +1449,54 @@ Reinstall it with the new release: `pie install flow-php/flow-php-ext`.
| `array_expand()` in `filter()`, `until()`, `duplicateRow()`, `aggregate()`, `over()` | throws `InvalidArgumentException` at `schema()` / `run()` |
| `array_expand()` in an `onEach()` body | throws `InvalidArgumentException` when `onEach()` is called |

### 101) `flow-php/array-dot` - `\` escapes every path grammar character, `array_dot_steps()` deprecated

| Before | After |
|-----------------------------------------------------|-----------------------------------------------|
| `\?x`, `a\*b`, `a\\` read keys `\?x`, `a\*b`, `a\\` | read keys `?x`, `a*b`, `a\` |
| `?{a}` reads key `{a}` | throws `InvalidPathException` - `?\{a\}` |
| `x.*.a\.b` reads `a` -> `b` of each element | reads key `a.b` of each element |
| `x.{a\.b,c}` result keys `a__ESCAPED_DOT__b`, `c` | `a.b`, `c` |
| `x.{a?b}` result key `ab` | `a?b` |
| `array_dot_steps($path)` | deprecated - `Path::fromString($path)->steps` |

### 102) `flow-php/array-dot` - `array_dot_set()` and `array_dot_rename()` change only the addressed key

| Before | After |
|---------------------------------------------------------------------------------------------|---------------------------------|
| `array_dot_set(['a' => ['x' => 1]], 'a.y', 2)` -> `['a' => ['y' => 2]]` | `['a' => ['x' => 1, 'y' => 2]]` |
| `array_dot_set([5 => 'a', 7 => 'b'], '5', 'c')` -> `['a', 'b', 'c']` | `[5 => 'c', 7 => 'b']` |
| `array_dot_set([], '\{a\}', 1)` / `'?a'` write keys `\{a\}` / `?a` | write keys `{a}` / `a` |
| `array_dot_set($array, 'x.*', $value)` throws | sets every element of `x` |
| `array_dot_set($array, 'x.{a,b}', $value)` writes key `{a,b}` | throws `InvalidPathException` |
| `array_dot_rename(['{a}' => 1], '\{a\}', 'b')` -> warning, `['{a}' => 1, 'b' => null]` | `['b' => 1]` |
| `array_dot_rename(['a' => 1], '?missing', 'b')` -> warning, `['a' => 1, 'b' => null]` | `['a' => 1]` |
| `array_dot_rename($array, 'x.*', 'c')` / `'x.{a,b}'` -> warning, adds `c => null` under `x` | throws `InvalidPathException` |
| `array_dot_set(['x' => 5], 'x.*', 1)` throws `InvalidPathException` | `['x' => []]` |
| `array_dot_set([], 'x.*', 1)` throws `InvalidPathException` | `['x' => []]` |
| `array_dot_rename(['a' => 1, 'b' => 2], 'a', 'a')` -> `['b' => 2]` | `['a' => 1, 'b' => 2]` |

### 103) `flow-php/etl` - `array_get_collection()` reads its keys as literal keys

| Before | After |
|----------------------------------------------|-----------------------------------|
| key `a.b` reads `a` -> `b`, result key `a_b` | reads key `a.b`, result key `a.b` |
| key `k,l` throws `InvalidArgumentException` | reads key `k,l` |

### 104) `flow-php/etl` - `array_expand()` over a null list gives no rows

| Before | After |
|-----------------------------------------------|----------------------|
| `null` list throws `InvalidArgumentException` | no rows for that row |

### 105) `flow-php/array-dot`, `flow-php/etl` - a nullsafe multimatch over an empty array reads `null`

| Before | After |
|---------------------------------------------------------------------------------------------------------|------------------------------------|
| `array_dot_get([], '{?a}')` throws `InvalidPathException` | `['a' => null]` |
| `array_dot_get([], '{a}')` message `Path "{a}" does not exists ...` | `Path "a" does not exists ...` |
| `array_get_collection(ref('c'), ['id'])` over `[['name' => 'a'], []]` throws `InvalidArgumentException` | `[['id' => null], ['id' => null]]` |

---

## Upgrading from 0.42.x to 0.43.x
Expand Down
29 changes: 29 additions & 0 deletions src/core/etl/src/Flow/ETL/DSL/functions.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -721,12 +721,32 @@ function when(mixed $condition, mixed $then, mixed $else = null): When
return new When($condition, $then, $else);
}

/**
* Alias for `array_get`.
*/
#[DocumentationDSL(module: Module::CORE, type: DSLType::SCALAR_FUNCTION)]
function structure_get(ScalarFunction $ref, string $path): ArrayGet
{
return array_get($ref, $path);
}

#[DocumentationDSL(module: Module::CORE, type: DSLType::SCALAR_FUNCTION)]
function array_get(ScalarFunction $ref, string $path): ArrayGet
{
return new ArrayGet($ref, $path);
}

/**
* Alias for `array_get_collection`.
*
* @param array<array-key, mixed>|ScalarFunction $keys
*/
#[DocumentationDSL(module: Module::CORE, type: DSLType::SCALAR_FUNCTION)]
function structure_get_collection(ScalarFunction $ref, ScalarFunction|array $keys): ArrayGetCollection
{
return array_get_collection($ref, $keys);
}

/**
* @param array<array-key, mixed>|ScalarFunction $keys
*/
Expand All @@ -736,6 +756,15 @@ function array_get_collection(ScalarFunction $ref, ScalarFunction|array $keys):
return new ArrayGetCollection($ref, $keys);
}

/**
* Alias for `array_get_collection_first`.
*/
#[DocumentationDSL(module: Module::CORE, type: DSLType::SCALAR_FUNCTION)]
function structure_get_collection_first(ScalarFunction $ref, string ...$keys): ArrayGetCollection
{
return array_get_collection_first($ref, ...$keys);
}

#[DocumentationDSL(module: Module::CORE, type: DSLType::SCALAR_FUNCTION)]
function array_get_collection_first(ScalarFunction $ref, string ...$keys): ArrayGetCollection
{
Expand Down
3 changes: 1 addition & 2 deletions src/core/etl/src/Flow/ETL/Function/ArrayExpand.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@

namespace Flow\ETL\Function;

use Flow\ETL\Exception\InvalidArgumentException;
use Flow\ETL\Exception\SchemaNotDerivableException;
use Flow\ETL\FlowContext;
use Flow\ETL\Function\ScalarFunction\ExpandResults;
Expand Down Expand Up @@ -89,7 +88,7 @@ public function eval(Row $row, FlowContext $context): array
$array = (new Parameter($this->ref))->asArray($row, $context);

if ($array === null) {
throw new InvalidArgumentException('ArrayExpand requires non-null array');
return [];
}

if ($this->expand === ArrayExpand\ArrayExpand::KEYS) {
Expand Down
37 changes: 26 additions & 11 deletions src/core/etl/src/Flow/ETL/Function/ArrayGet.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,32 +4,42 @@

namespace Flow\ETL\Function;

use Flow\ArrayDot\Exception\InvalidPathException;
use Flow\ArrayDot\Path;
use Flow\ArrayDot\Step\Key;
use Flow\ETL\Exception\InvalidArgumentException;
use Flow\ETL\Exception\SchemaNotDerivableException;
use Flow\ETL\FlowContext;
use Flow\ETL\Row;
use Flow\Types\Type;
use Flow\Types\Type\ArrayKey;
use Flow\Types\Type\Logical\StructureType;
use Flow\Types\Type\TypeWidener;

use function explode;
use function Flow\ArrayDot\array_dot_get;
use function Flow\Types\DSL\type_bare;
use function Flow\Types\DSL\type_instance_of;
use function sprintf;
use function str_contains;

final class ArrayGet implements ScalarFunction
{
use ScalarFunctionChain;

private readonly Path $path;

public function __construct(
private readonly ScalarFunction $ref,
private readonly string $path,
string $path,
) {
// A wildcard path produces N values from runtime keys - not a single column with one type.
if (str_contains($path, '*') || str_contains($path, '{')) {
try {
$this->path = Path::fromString($path);
} catch (InvalidPathException $e) {
throw new InvalidArgumentException(sprintf('ArrayGet path "%s" is not a valid path.', $path), 0, $e);
}

if (!$this->path->selectsSingleValue()) {
throw new InvalidArgumentException(sprintf(
'ArrayGet path "%s" contains a wildcard - a wildcard path cannot describe a single column. Use array_get_collection() instead.',
'ArrayGet path "%s" selects more than one value - only a path of keys describes a single column. Use array_get_collection() instead.',
$path,
));
}
Expand All @@ -49,7 +59,7 @@ public function children(): array
public function withChildren(array $children): static
{
/** @var list<ScalarFunction> $children */
return new self($children[0], $this->path);
return new self($children[0], $this->path->toString());
}

/**
Expand All @@ -58,8 +68,11 @@ public function withChildren(array $children): static
public function returns(): Type
{
$type = type_bare($this->ref->returns());
$nullable = false;

foreach (explode('.', $this->path) as $segment) {
foreach ($this->path->steps as $step) {
// the constructor accepts a path of keys only
$key = type_instance_of(Key::class)->assert($step);
$bare = type_bare($type);

if (!$bare instanceof StructureType) {
Expand All @@ -70,19 +83,21 @@ public function returns(): Type
}

// '0' finds the element named int 0, exactly as it did when elements were array keys
$element = $bare->element(ArrayKey::coerce($segment));
$element = $bare->element(ArrayKey::coerce($key->name));

if ($element === null) {
throw SchemaNotDerivableException::function(
'array_get',
'path segment "' . $segment . '" is not declared by "' . $bare->toString() . '"',
'path segment "' . $key->name . '" is not declared by "' . $bare->toString() . '"',
);
}

// a nullsafe step reads null where an optional element is absent
$nullable = $nullable || $key->nullsafe && $element->optional;
$type = $element->type;
}

return $type;
return $nullable ? (new TypeWidener())->nullable($type) : $type;
}

public function eval(Row $row, FlowContext $context): mixed
Expand Down
Loading