Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • specifications/tr/tr-0070
1 result
Show changes
Commits on Source (7)
Hide whitespace changes
Inline Side-by-side
.gitignore 0 → 100644
View file @ 34534088
.DS_Store
TR-0070-Enhanced-filtering-and-queries.md
View file @ 34534088
![oneM2M logo](media/logo.png)
|oneM2M<br />Technical Report |oneM2M<br />Technical Report |
|-|-|
|Document Number |TR-00xx-V-x.y.z |
|Document Name: |The name of the document<br /> |
|Date: |&lt;20yy-mm-dd> |
|Abstract: |&lt; An abstract of the document and information that may be used in subsequent electronic searches> |
|Document Number |TR-0070-V-0.0.1 |
|Document Name: |Enhanced filtering and Queries |
|Date: |<mark>2024-08-22</mark> |
|Abstract: |This TR provides an overview about the current status of querying and filtering in oneM2M, formulates new use cases, and provides an overview about relevant query languages. |
|Template Version: January 2020 (do not modify) | |
......@@ -45,55 +44,28 @@ NO REPRESENTATION OR WARRANTY IS MADE THAT THE INFORMATION IS TECHNICALLY ACCURA
# Contents
[Contents](#contents)
[1 Scope](#1-scope)
[2 References](#2-references)
&nbsp;&nbsp;&nbsp;&nbsp;[2.1 Normative references](#21-normative-references)
&nbsp;&nbsp;&nbsp;&nbsp;[2.2 Informative references](#22-informative-references)
[3 Definition of terms, symbols and abbreviations](#3-definition-of-terms,-symbols-and-abbreviations)
&nbsp;&nbsp;&nbsp;&nbsp;[3.1 Terms](#31-terms)
&nbsp;&nbsp;&nbsp;&nbsp;[3.2 Symbols](#32-symbols)
&nbsp;&nbsp;&nbsp;&nbsp;[3.3 Abbreviations](#33-abbreviations)
[4 Conventions](#4-conventions)
[5 User defined clause(s) from here onwards](#5-user-defined-clause%28s%29-from-here-onwards)
&nbsp;&nbsp;&nbsp;&nbsp;[5.1 User defined subdivisions of clause(s) from here onwards](#51-user-defined-subdivisions-of-clause%28s%29-from-here-onwards)
[Proforma copyright release text block](#proforma-copyright-release-text-block)
&nbsp;&nbsp;&nbsp;&nbsp;[Annexes](#annexes)
[Annex &lt;A>:Title of annex](#annex-%26lta%3E%3Atitle-of-annex)
[Annex &lt;B>:Title of annex](#annex-%26ltb%3E%3Atitle-of-annex)
&nbsp;&nbsp;&nbsp;&nbsp;[ First clause of the annex](#----first-clause-of-the-annex)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[B.1.1 First subdivided clause of the annex](#b11----first-subdivided-clause-of-the-annex)
[Annex &lt;y>:Bibliography](#annex-%26lty%3E%3Abibliography)
[History](#history)
# 1 Scope
The present document ...
`EXAMPLE: The present document provides the necessary adaptions to the endorsed document.`
<mark>The Scope shall not contain requirements.</mark>
# 1 Scope
This document presents possible requirements for an enhanced filtering and query mechanism as well as a discussion about possible solutions.
# 2 References
<mark>The following text block applies.</mark>
References are either specific (identified by date of publication and/or edition number or version number) or nonspecific. For specific references,only the cited version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies.
## 2.1 Normative references
<mark>As a Technical Report (TR) is entirely informative it shall not list normative references.</mark>
The following referenced documents are necessary for the application of the present document.
Not applicable.
- <a name="_ref_1">[1]</a> TS-0001: "Functional Architecture"
- <a name="_ref_2">[2]</a> TS-0002: "Requirements"
## 2.2 Informative references
<mark>Clause 2.2 shall only contain informative references which are cited in the document itself.</mark>
The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area.
- <a name="_ref_i.1">[i.1]</a> oneM2M Drafting Rules [https://member.onem2m.org/static_Pages/others/Rules_Pages/oneM2M-Drafting-Rules-V1%202%202.doc](https://member.onem2m.org/static_Pages/others/Rules_Pages/oneM2M-Drafting-Rules-V1%202%202.doc)
- <a name="_ref_i.2">[i.2]</a> Wikipedia entry on "S-expressions" [https://en.wikipedia.org/w/index.php?title=S-expression&oldid=1178440297](https://en.wikipedia.org/w/index.php?title=S-expression&oldid=1178440297)
# 3 Definition of terms, symbols and abbreviations
......@@ -101,94 +73,513 @@ The following referenced documents are not necessary for the application of the
## 3.1 Terms
<mark>Clause numbering depends on applicability.</mark>
For the purposes of the present document, the following terms and definitions apply:
<mark>TODO</mark>
- <mark>A definition shall not take the form of, or contain, a requirement.</mark>
- <mark>The form of a definition shall be such that it can replace the term in context. Additional information shall be given only in the form of examples or notes (see below).</mark>
- <mark>The terms and definitions shall be presented in alphabetical order.</mark>
SPARQL: SPARQL is a semantic query language to retrieve and manipulate RDF data. It is a W3C specification.
For the purposes of the present document, the [following] terms and definitions [given in ... and the following] apply:
## 3.2 Symbols
<mark>Definition format</mark>
N/A
<mark>&lt;defined term>: &lt;definition></mark>
## 3.3 Abbreviations
<mark>If a definition is taken from an external source, use the format below where [N] identifies the external document which must be listed in Section 2 References.</mark>
For the purposes of the present document, the following abbreviations apply:
<mark>&lt;defined term>[N]: &lt;definition></mark>
<mark>TODO</mark>
<mark>example 1: text used to clarify abstract rules by applying them literally</mark>
CSE: Common Services Entity
> NOTE: This may contain additional information.
IMEI: International Mobile Equipment Identity
## 3.2 Symbols
JSON: JavaScript Object Notation
<mark>Clause numbering depends on applicability.</mark>
RDF: Resource Description Framework
For the purposes of the present document, the [following] symbols [given in ... and the following] apply:
SQL: Structured Query Language
<mark>Symbol format</mark>
XML: Extensible Markup Language
<mark>`&lt;symbol> &lt;Explanation>`</mark>
<mark>`&lt;2nd symbol> &lt;2nd Explanation>`</mark>
<mark>`&lt;3rd symbol> &lt;3rd Explanation>`</mark>
XSD: XML Schema Definition
## 3.3 Abbreviations
# 4 Conventions
<mark>Abbreviations should be ordered alphabetically.</mark>
The key words "Shall", "Shall not", "May", "Need not", "Should", "Should not" in this document are to be interpreted as described in the oneM2M Drafting Rules <a href="#_ref_i.1">[i.1]</a>
<mark>Clause numbering depends on applicability.</mark>
# 5 Introduction
For the purposes of the present document, the [following] abbreviations [given in ... and the following] apply:
This clause provides an overview about the current filter am query methods in oneM2M.
<mark>Abbreviation format</mark>
## 5.1 Current Method of Filtering in oneM2M
<mark>`&lt;ABBREVIATION1> &lt;Explanation>`</mark>
<mark>`&lt;ABBREVIATION2> &lt;Explanation>`</mark>
<mark>`&lt;ABBREVIATION3> &lt;Explanation>`</mark>
The current method for providing filters and queries in oneM2M requests is currently done by providing **_FILTER CRITERIA_** in a request. This is a complex attribute that contains a structure of matching conditions attributes and filter handling attributes that can be applied for certain RETRIEVE and DISCOVERY operations. The detailed description and the applicability is specified in TS-0001, clause 8.1.2 and table 8.1.2-2.
# 4 Conventions
To summarise its use:
The key words "Shall", "Shall not", "May", "Need not", "Should", "Should not" in this document are to be interpreted as described in the oneM2M Drafting Rules <a href="#_ref_i.1">[i.1]</a>
- *Matching conditions* are evaluated against resources. When the evaluation is true, the matched resources are part of the matching result.
- The *filter handling conditions* provide additional input used to determine the filtering result (e.g. the maximum number of resources to be included in the filtering result).
- The *filtering result* may be composed of zero, one or more resources and determines the target resources for Discovery-based operations.
Further details are also provided in TS-0001 [[1](#_ref_1)] , clause 10.2.6 "Discovery-related Procedures".
# 5 User defined clause(s) from here onwards
&lt;Text>
The *matching conditions* is a set of attributes that specify for some common and universal resource attributes matching (or comparison) a value and the kind of comparison that should be applied if present. For example, the value of the *createdBefore* attribute is compared against the *creationTime* attribute during matching, and it will be matched if a resource's *createdTime* attribute is before the timestamp specified in *createdBefore*. Through the specific nature of the attributes it is possible to compare against the respective attributes in resources not only for equality, but also for smaller, larger, before, after, etc. However, these attribute-specific comparisons are only defined for a limited set of resource attributes.
Besides the pre-defined set of **_FILTER CRITERIA_** attributes it is possible to compare all these attributes attributes, and also the custom attributes in &lt;flexContainer> specialisations, against equality. However, equality is the only comparison type for most of oneM2M's attributes.
## 5.1 User defined subdivisions of clause(s) from here onwards
&lt;Text>
It is possible to add multiple queries and to logical combine them with setting the *filterOperations* **_FILTER CRITERIA_** attribute. This attribute specifies the logical operator that is used for all the given filter expressions. No sub-queries are allowed, eg. to mix different boolean operations. Negation of query results is also not possible.
<mark>The following text is to be used when appropriate:</mark>
In addition **_FILTER CRITERIA_** *filter handling* attributes are used to limit the number of filter result, specifying an offset in the filter result, or limit the search depth in the resource tree.
TS-0004 [[1](#_ref_1)] , clause 7.3.3.17.16 "Conditions on labelsQuery attribute" defines a tiny query language to do add basic comparison operators to handle filtering on a resource's *label* attribute (which is a list and may contain multiple tags, or tokens). However, this filter query is defined only for the *labels* attribute, and not for other scalar or list-type attributes.
# <mark>Proforma copyright release text block</mark>
<mark>This text box shall immediately follow after the heading of an element (i.e. clause or annex) containing a proforma or template which is intended to be copied by the user. Such an element shall always start on a new page.</mark>
<mark>TODO: Example</mark>
<mark>Notwithstanding the provisions of the copyright clause related to the text of the present document, oneM2M grants that users of the present document may freely reproduce the &lt;proformatype> proforma in this {clause|annex} so that it can be used for its intended purposes and may further publish the completed &lt;proformatype>.</mark>
## 5.2 Semantic Discovery and Query
<mark>&lt;PAGE BREAK></mark>
Both semantic discovery and query are not part of this document because they use a different method for handling queries, but they still should be shortly summarised here because they are an example of an additional query method in oneM2M that work together with the methods provided by the ***Filter Criteria*** request attribute.
Semantic queries are different from the filter method described in clause [5.1](#51-current-method-of-filtering-in-onem2m). The filtering methods described in that clause are part of the request handling. A result set of resources or references is created or filtered afterwards according to the given configuration in the **_FILTER CRITERIA_** request attribute.
## <mark>Annexes</mark>
<mark>Each annex shall start on a new page (insert a page break between annexes A and B, annexes B and C, etc.).<mark>
For semantic discovery and query two different methods are defined that use semantic service components to compute the result set according not only by just comparing queries and values, but by deriving explicit and implicit information from the semantic information that are provided together with the resources as well as from external information that are semantically referenced.
# Annex &lt;A>:<br />Title of annex
Resources may have child resources of type &lt;semanticDescriptor>, which contain additional semantic information that may describe a resource respective the data contained in that resource further. For example, the data from a temperature sensor may be annotated this way to explain that data producing device is a thermostat used indoor in an industrial context, and that the data is in degree Celsius. The same semantic method can be used to semantically describe the device itself.
The differences between semantic discovery and semantic query operations are:
- A *semantic discovery* uses the information stored in the &lt;semanticDescriptor> child-resources together with other information to build a result set for a semantic expression. For example, "discover and return all the temperature data from devices that are used indoor".
- A *semantic query* is used to send and process a semantic expression by the semantic service component of the CSE. It may use all the available semantic information from the resources in the resource tree. The result, however, is not a set of resources or resource references, but a semantic result string (usually RDF triples).
The query format used in both cases is SPARQL.
## 5.3 Shortcomings of the Current Filtering Method
The current filtering method described in clause [5.1](#51-current-method-of-filtering-in-onem2m) has limitations that prevents to run more sophisticated RETRIEVE and DISCOVERY requests in the CSE. This means that an AE that needs more elaborate filtering capabilities has to implement them itself, which in case of a restricted device is not always easy or even possible.
To summarise the limitations:
- Comparison operations such as "greater", "less", "earlier", "later" are defined for only a limited set of common and universal attributes.
- Other resource attributes that are not part of this predefined set, including &lt;flexContainer> custom attributes, can only be compared for equality.
- Filtering of attributes that are part of complex attributes is not possible.
- Filtering on elements of a list or sub-strings in a string is not possible.
- Filter expressions that contain multiple attributes can only be combined via "AND", "OR", or "XOR" logical operators.
- Filter expressions cannot be negated (logical operator "NOT")
## 5.4 Evaluation Criteria of &lt;action>, &lt;dependency>, and &lt;state> Resource Types
In addition to the queries and filters provided in requests some resources also use a similar method to perform their tasks. The &lt;action> and &lt;dependency> resources provide a simple method to react on changes in other resources without notifying an Application Entity to perform the necessary requests itself. In the resource type &lt;state> the *evalCriteria* attribute is used to control state transitions.
These resource types have attributes that contain an evaluation criteria that is used for the comparison as soon as an update happens to a target resource. This is sufficient for simple scenarios, such as switching on and off a light depending on the value of an updated *content* attribute in a &lt;container> resource, but more sophisticated scenarios, for example evaluating multiple or complex attributes, are not possible.
To summarise the limitations:
- It is only possible to evaluate a single "top-level" attribute of a subject resource.
- The comparison operations are limited to a few basic comparison operations (e.g. "in" to test for the existence in a list is not possible).
# 6 Requirements for IoT & oneM2M Resource Querying and Filtering
This section provides discussions on existing an new requirements for querying and filtering.
## 6.1 Existing Requirements in TS-0002
The following requirements regarding resource querying and filtering can be found in oneM2M TS-0002 [[2](#_ref_2)].
**Table 6.1-1: List of existing requirements in TS-0002**
| Requirement ID | Description | Release |
| ----------------------------------- | ------------------------------------------------------------ | --------------- |
| OSR-083<br /> See REQ-2015- 0593R02 | The oneM2M System shall be able to filter information from oneM2M Devices for a given set of parameters. | Not implemented |
| ONT-018<br /> See REQ-2018- 0057R01 | The oneM2M system shall support semantic query and discovery across heterogeneous ontologies including support of automatic ontology mapping and semantic reasoning. | 4 |
| ONT-018<br /> See REQ-2018- 0057R01 | The oneM2M system shall support semantic query and discovery across heterogeneous ontologies including support of automatic ontology mapping and semantic reasoning. | 4 |
The first requirement implies a technical filter or query while the other requirements are related to semantic querying and discovery.
## 6.2 New Use Cases
The following is a list of new use cases that are interesting for IoT application as well as for additional functionality for common services within the CSE.
### 6.2.1 Searching for Specific Protocol Schemes in *pointOfAccess* Attribute
A management system (Application Entity) needs to find out which other Application Entities are request reachable through a specific protocol, e.g. *MQTT*. For this, the Application Entity performs a query to search for the sub-string "mqtt://" in the entries of the *pointOfAccess* attribute of all resources with the *resourceType* "Application Entity".
**Requirement(s)**
- Filter on a sub-string in list attributes.
### 6.2.2 Monitor Temperature Values
#### 6.2.2.1 Compare Using &lt;flexContainer> Resources
An application entity wants to retrieve only those [temperature] &lt;flexContainer> specialisations where the *currentTemperature* attribute is above a certain value.
**Requirement(s)**
- Filter on a custom attribute that is greater than a comparison value.
#### 6.2.2.2 Compare Using &lt;container> and &lt;contentInstance> Resources
An application entity wants to retrieve only those &lt;container> resources where any of the child &lt;contentInstance> resources contains a temperature above a certain value.
**Requirement(s)**
- Filter on the values of attributes in child resources.
### 6.2.3 Search for a Tag in *label* OR Where Different Attributes are Larger Than a Specific Values
An application entity needs to retrieve resources that have specific a tag present in a *label* attribute, OR those &lt;flexContainer> specialisations where custom attributes *a* AND *b* are BOTH larger than specific values.
**Requirement(s)**
- Filter on a single tag in a list attribute.
- Filter on multiple attributes that are greater than specific values.
- Combine comparison expressions with OR and AND.
### 6.2.4 Search for Subscriptions that Monitor a Certain Child-Resource Type
An application entity needs to retrieve all the &lt;subscription> resources that monitor &lt;contentInstance> resources. This involves a comparison to the *childResourceType* list attribute of the *eventNotificationCriteria* complex attribute.
**Requirement(s)**
- Filter on an attribute that is part of a complex type (child-attribute of another attribute).
### 6.2.5 Reacting on Multiple Attribute Changes for &lt;action> Resources
An application entity wants to install an &lt;action> resource that monitors a flexContainer> specialisation with attributes *a* and *b*. If both attributes in that resource rises above a certain value, the &lt;action> resource shall assign a value to an attribute in another resource.
Similar requirements can be deducted for use of the &lt;dependency> and &lt;state> resource types.
**Requirement(s)**
- Be able to evaluate one or more attributes in one evaluation criteria.
- Be able to evaluate complex type attributes.
- Support the same comparison operators as in other queries (e.g. "in")
---
<mark>TODO: Use case for parent- or child-resource attributes</mark>
## 6.3 Requirements
The following requirements are derived from the comparison above. In addition, some non-functional requirements are formulated.
### 6.3.1 Functional Requirements
- Being able to compare custom and resource specific attributes
- Being able to use comparison operators other than only equality, such as lesser and greater than
- Support other operators such as "in" for lists and strings
- Provide sub-queries
- Being able to mix boolean operations for individual or sub-queries
- Allow to reference resource's parent and child resources and user their attributes in comparisons as well.
- Provide the same functionality for &lt;action> and &lt;dependency> Resources
- <mark>TBC</mark>
### 6.3.2 Non-Functional Requirements
- Ease-of-use
- Simplicity
- Size of implementation
- Be backward compatible with the existing ***Filter Criteria***, and ideally can be combined
- <mark>TBC</mark>
# 7 Query Languages
This section provides an overview about some existing query languages.
## 7.1 Query Languages Comparison
Data query languages are syntaxes designed specifically for querying and retrieving data from databases or data repositories. These query languages provide a way to interact with data, making it easier for applications to retrieve the information they need. Here is a short summary of relevant data query languages:
1. **SQL (Structured Query Language)**: SQL is the most widely used query language for relational databases. It allows users to retrieve, manipulate, and manage data stored in tables. SQL uses a declarative approach, where applications specify what data they want to retrieve rather than how to retrieve it. SQL supports various operations like SELECT, INSERT, UPDATE, and DELETE, enabling users to perform complex queries and aggregations on structured data. Relevant for filtering is the WHERE clause where filter expressions respective query statements can be formulated.
2. **XQuery**: XQuery is a query language specifically designed for querying XML (eXtensible Markup Language) data. It allows users to extract and manipulate data from XML documents using a variety of query and transformation capabilities. XQuery supports XPath, a language for navigating XML structures, making it suitable for querying and transforming complex XML data.
3. **JSON Query** (also known as JSONPath) is a syntax used to query and extract specific data from JSON documents provides a way to navigate through the hierarchical structure of JSON data and select elements based on specific criteria. It uses a path-like syntax to traverse the JSON structure and select elements or values.
### 7.2 Query Languages Examples
> 📝 Could also be moved to annex
The following examples show queries for the query languages introduced before.
#### 7.2.1 SQL
The following example queries data records from an addresses table filtered by a specific street name.
```sql
SELECT *
FROM addresses
WHERE street_name = 'Main Street';
```
The following example queries data records from an addresses table filtered by a specific street name and street number.
```sql
SELECT *
FROM addresses
WHERE street_name = 'Main Street' AND street_number < 10;
```
The following example queries structured data records from an addresses table by a specific street name.
```sql
SELECT address.street
FROM addresses
WHERE addresses.address.street_number < 10;
```
#### 7.2.2 XQuery
In general, XML and XQuery support organization of and access to complex structured data.
The following example queries elements from an XML document filtered by a specific street name.
```XQuery
for $address in /addresses/address
where $address/street_name = 'Main Street'
return $address
```
The following example queries elements from an XML document filtered by a specific street name and street number.
```xquery
for $address in /addresses/address
where $address/street_name = 'Main Street' and $address/street_number < 10
return $address
```
#### 7.2.3 JSON Query
In general, JSON and JSON Query support organization of and access to complex structured data.
The following example queries elements
```json
$.address[?(@.streetName == "Main Street")].streetName
```
```json
$.address[?(@.streetNumber < 10 && @.streetName == "Main Street")].streetName
```
## 7.3 Alternatives to Improve oneM2M's Filters and Queries
The data query languages in section <mark>7.1</mark> provide efficient and straightforward methods for retrieving and manipulating data from databases or data sources. They offer different syntaxes and support the specific data formats and use cases and simplifying data retrieval.
However, they are tailored to the specific data formats they are designed to support. oneM2M uses XSD, the XML Schema Definition language, to formally define its specification's artefacts (for example the resources and request structures). It would seem that *XQuery* would be a natural choice for a query language in oneM2M. However, it shares some drawbacks with the other presented query languages:
- The presented query languages are more suited for the environments for which they have been designed. oneM2M defines its own data model which is then mapped to either XML, JSON or CBOR when exchanging data with an Application Entity or another CSE during a request/response exchange.
- Though CSE implementation may choose to support any of these encoding formats, they usually only support one or two of them. Adding any of the frameworks to an implementation would probably increase the size of an implementation considerably. This would especially be the case for smaller implementations that would run as an ASN on a constraint device. Those implementations may choose only to support one content encoding (XML, JSON or cbor). Adding a query frameworks that originates from any of the the other, unsupported encodings would add even more size to a deployment.
## 7.3.1 Enhancing the Existing *Filter Criteria* Method
The main limitation of the current filtering in oneM2M is the lack of recursivity, which forbids to express complex requests, and the fact that only the OR logical operator can be used between matching conditions of the same type (i.e. attributes).
#### 7.3.1.1 Filter Groups
This first proposal therefore focuses on adding a new *filterGroup (fg)* matching condition that allows nesting expressions and enforces operations pointed to by the *filterOperation (fo)* tag also between Filter Matching Conditions of the same type. Making the operation type defined in *filterOperation* tag valid for all the Filter Matching Conditions both makes the syntax more clear/readable and allows using all AND/OR/XOR/NOT operations also for attributes.
With this approach it is not only possible to apply different operations between attribute tags in general but also to use different operations for each subsequent *filterGroup* when needed.
Example:
```xml
<filterCriteria>
<fo>2</fo>
<attribute> … </attribute>
<fg>
<fo>1</fo>
<labels> … </labels>
<childLabels > … </childLabels>
</fg>
<fg>
<fo>1</fo>
<attribute> … </attribute>
<fg>
<fo>2</fo>
<labels> … </labels>
<childLabels> … </childLabels>
</fg>
</fg>
</filterCriteria>
```
This example displays new possibilities to express complex requests:
- Multiple instances of *filterGroup* matching condition allowed.
- *filterGroup* matching conditions can be nested.
- Separate filtering operation defined for each *filterGroup* matching condition.
> **Note**
> It is also proposed to remove a quite confusing rule in TS-0004 7.3.3.17.0, saying that:
> "If multiple matching conditions of the same type (i.e. same condition tag) are present in the Filter Criteria parameter, these shall be combined by applying logical OR operation".
With the proposed extension,
- to send a disjunction request with multiple matching conditions of the same type, the *filterOperation* must be explicitely set to OR. Though this brings a backward incompatibility, this is limited to matching conditions that originally had 0..n multiplicity, that is: *contentType*, attributes, childAttributes, parentAttributes, *semanticsFilter*.
- it is allowed to use conjonctions (and XOR operations) requests with multiple matching conditions of the same type, for instance on attributes (e.g. *attr1=val1 and attr2=val2*), which was not possible before.
#### 7.3.1.5 Advantages and Disadvantages
- Major increase in semantic expressiveness with minimal syntactic change: the syntax is the same as the existing one, with only one new filter criterium; basically, it just allows nesting expressions of the existing request language.
- Easy to implement, for the same reason.
- A logical *NOT* operation is added inside the *filterGroup*.
<br />
- It does not allow non-equality operations on attributes (it could be feasible, but complex to implement and costly at runtime).
- It does not extend path-based requests à la XPath/JSONPath (except current child/parent equality on labels and attributes).
- Possible break of backward compatibility if the rule on the default value of *fo* is removed (see the note above).
## 7.3.2 Defining a Simplified New Filter and Query Language
The input contribution SDS-2022-0014R01 proposes to add a new attribute *advancedQuery* to ***Filter Criteria*** that contains a filter string with a syntax based on S-expressions <a href="#_ref_i.2">[i.2]</a>. The new attribute, *advancedQuey*, can be used together in combination with the already existing attributes in ***Filter Criteria*** or alone.
It is an extension of the proposal in clause 7.3.1; the *filterGroup* instances are move from a list attribute to S-expressions.
#### 7.3.2.1 S-expressions
S-expressions are a notation that is based on a nested list (or tree) structure. Those lists may contain either none, one or many elements. Each of the elements can either be value or another S-expression. The first element can also be a function which takes the other following elements of the list as input (after recursively evaluating them). S-expressions are the basis for lisp-like languages, like *Common Lisp*, *Scheme*, or *Closure*.
Examples:
- `(1 2 3)` : This S-expression is a list with the three values 1, 2, 3
- `(== 1 2)` : This S-expression evaluates the function `==` with the parameters `1` and `2`. It evaluates to `false`
- `(| (== 1 2) (< 3 4))` : This S-expression evaluates the function `|` (or) with the result of the two S-Expressions `(== 1 2)` (that evaluates to *false*) and `(< 3 4` (that evaluates to *true*). The whole S-expression evaluates to `true`.
The advantage of S-expressions are that parsers and interpreters are fairly simple to implement in any programming language, and that implementations have a small footprint.
#### 7.3.2.2 Supported Comparisons
Since S-expressions are evaluated recursively no loop or expression functionalities need to be supported. The number of operations can be limited to a small set of logical and comparison operations. The following list can be extended if needed.
The S-expression for queries and filters in oneM2M has at least one operator followed by an attribute name or an S-expression, optionally followed by one or more values or S-expressions. Attribute names are replaced by their respective values from the evaluated resources during a query:
```(<operator> <attribute name or S-expression> [<value or S-expression>]*)```
**Table 7.3.2.2-1: Supported Logical Operators**
| Operator | Description | Examples |
| -------- | ------------------------------------------------------------ | --------------------------------- |
| & | Logical *and*. All included expressions must evaluate to *true* for the expression to evaluate to *true*, otherwise the expression evaluates to *false*. | `(& (== ty 1) (in lbl "aLabel"))` |
| \| | Logical *or*. At least one of the included expressions must evaluate to *true* for the expression to evaluate to *true*, otherwise the expression evaluates to *false*. | `(\| (== ty 1) (== ty 2))` |
| ! | Logical *not*. Only one included expression is allowed, which boolean value is negated as a result of the evaluation. | `(! (== ty 1))` |
**Table 7.3.2.2-2: Supported comparison Operators**
| Operator | Description | Examples |
| -------- | ------------------------------------------------------------ | --------------- |
| == | Equality comparison. All included and evaluated expressions are compared to each other. <br />Only if all included expressions are equal to each other the result of the comparison is *true*, otherwise the expression evaluates to *false*. | `(== ty 1)` |
| != | Inequality comparison. All included and evaluated expressions are compared to each other. <br />Only if all included expressions are not equal to each other the result of the comparison is *true*, otherwise the expression evaluates to *false*. | `(!= ty 1)` |
| < | Less-than comparison. All included and evaluated expressions are compared to each other in order.<br />Only if all included expressions are monotonically increasing the result of the comparison is *true*, otherwise the expression evaluates to *false*. | `(< 1 2 3)` |
| <= | Less-than-or-equal comparison. All included and evaluated expressions are compared to each other in order. <br />Only if all included expressions are less than or equal to the value of their right expression the result of the comparison is *true*, otherwise the expression evaluates to *false*. | `(<= 1 2 2 3)` |
| > | Greater-than comparison. All included and evaluated expressions are compared to each other in order.<br />Only if all included expressions are monotonically decreasing the result of the comparison is *true*, otherwise the expression evaluates to *false*. | `(> 3 2 1)` |
| >= | Greater-than-or-equal comparison. All included and evaluated expressions are compared to each other in order. <br />Only if all included expressions are greater than or equal to the value of their right expression the result of the comparison is *true*, otherwise the expression evaluates to *false*. | `(>= 3 2 2 1)` |
| in | Contains comparison. The first expression must evaluate to a string or list. All remaining expressions must be contained in that string or list for the expression to evaluate to *true*, , otherwise the expression evaluates to *false*. | `(in (1 2 ) 2)` |
The following is a list of operators or functions for further extensions that may be included later if needed:
- startsWith, endsWith: compare only the start or beginning of a string
- regular expression matches
- case-insensitive comparisons for strings
- presence: an attribute is present or not, regardless of its value
#### 7.3.2.3 Attribute Paths
To support complex attributes and also parent-child relationships for queries the following syntax options for attribute paths are proposed:
**Table 7.3.2.3-1: Attribute paths syntax**
| Syntax | Description | Example |
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| `attribute` | A single top-level attribute of an evaluate resource | `rn`: ResourceName |
| attribute.sub-attribute<br />attribute.sub-attribute.sub-sub-attribute<br />... | A single attribute that is part of a complex structure. It can be further down the path of complex attributes | `loc.typ`: type of m2m:geoCoordinates |
| attribute{}sub-attribute<br />attribute.sub-attribute{}sub-sub-attribute<br />attribute{}sub-attribute{}sub-sub-attribute<br />... | A single attribute that is a member of a list attribute. During evaluation, all the members of a list are evaluated.<br />For a successful comparison of list attributes, only one occurrence in a list needs to successfully fulfil the comparison evaluation. | `pc.acr{}acod{}chty`: The *chty* attribute in *accessControlObjectDetails*. |
| cattr."attribute path" | Apply an attribute path to all the direct child-resources of the currently tested resource. All the direct child-resources are | |
| ^^."attribute path" | Apply an attribute path to the parent resource of a currently tested resource. | `^^.lbl`: The *labels*** attribute of the parent resource. |
> **Note**
>
> It is common in lisp-like languages to use the `-` character in function and attribute names.
If an attribute is not present in a resource, for example in case of an optional attribute, the expression returns `NULL`, which for a boolean expression evaluates to `False`.
#### 7.3.2.4 Functions and Iterators
Some functionality is needed to support iterations over direct child resources and comparing their attributes against a comparison. The following table lists those functions.
**Table 7.3.2.4-1: Functions and iterators**
| Function | Description | Example |
| ------------ | ------------------------------------------------------------ | --------------------------------------- |
| any-child | Iterate over every direct child resource of a currently tested resource and individually evaluate the argument S-expression. The function will evaluate to `true` if the S-expression evaluates to `true` for any of the direct child resources of the currently tested resource. This function can be applied recursively to include sub-child resources as well. | `(any-child (& (== ty 4) (> con "40"))` : Test if any &lt;contentInstance> has a value that is greater than "40". |
| all-children | Iterate over every direct child resource of a currently tested resource and individually evaluate the argument S-expression. The function will evaluate to `true` if the S-expresson evaluates to `true` for all the direct child resources of the currently tested resource. This function can be applied recursively to include sub-child resources as well. | `(all-children (== ty 3))` : Test if all the child resources are of type &lt;container>. |
#### 7.3.2.5 Advantages and Disadvantages
- An implementation for an S-expression parser and interpreter could be fairly small and would fit in smaller ASN and MN CSE implementations.
- The size of filter queries formulated as S-expressions is also small. This reduces the size of requests.
- The syntax is simple and easy to understand. The operators and functions can be defined specifically for by oneM2M purposes.
<br />
- There is no implementation, framework or library available.
- The specification of the syntax and the operators and functions needs to be maintained by oneM2M.
# 8 Conclusion
<mark>TODO</mark>
# Annexes
&lt;Text>
<mark>&lt;PAGE BREAK></mark>
# Annex &lt;B>:<br />Title of annex
# Annex &lt;A>:<br />Title of annex
&lt;Text>
## <!--B.1--> First clause of the annex
## <!--A.1--> First clause of the annex
&lt;Text>
### B.1.1 First subdivided clause of the annex
### A.1.1 First subdivided clause of the annex
&lt;Text>
......