|Abstract: |This TR provides an overview about the current status of querying and filtering in oneM2M, formulates new use cases, and provides an overview about relevant query languages. |
|Template Version: January 2020 (do not modify) | |
...
...
@@ -44,36 +44,6 @@ NO REPRESENTATION OR WARRANTY IS MADE THAT THE INFORMATION IS TECHNICALLY ACCURA
[5.1 Current Method of Filtering in oneM2M](#51-current-method-of-filtering-in-onem2m)
[5.2 Semantic Discovery and Query](#52-semantic-discovery-and-query)
[5.3 Shortcomings of the Current Filtering Method](#53-shortcomings-of-the-current-filtering-method)
[5.4 Evaluation Modes of <action> and <dependency> Resource Types](#54-evaluation-modes-of-%26ltaction%3E-and-%26ltdependency%3E-resource-types)
[6 Requirements for IoT & oneM2M Resource Querying and Filtering](#6--requirements-for-iot-%26-onem2m-resource-querying-and-filtering)
[6.1 Existing Requirements in TS-0002](#61-existing-requirements-in-ts-0002)
[6.2 New Use Cases](#62-new-use-cases)
@@ -165,7 +135,7 @@ In addition **_FILTER CRITERIA_** *filter handling* attributes are used to limi
## 5.2 Semantic Discovery and Query
Both semantic discovery and query are not part of this report because they use a different method for handling queries, but they still should be shortly summarised here because they are an example of an additional query method in oneM2M that work together with the methods provided by the ***Filter Criteria*** request attribute.
Both semantic discovery and query are not part of this document because they use a different method for handling queries, but they still should be shortly summarised here because they are an example of an additional query method in oneM2M that work together with the methods provided by the ***Filter Criteria*** request attribute.
Semantic queries are different from the filter method described in clause [5.1](#51-current-method-of-filtering-in-onem2m). The filtering methods described in that clause are part of the request handling. A result set of resources or references is created or filtered afterwards according to the given configuration in the **_FILTER CRITERIA_** request attribute.
...
...
@@ -183,7 +153,7 @@ The query format used in both cases is SPARQL.
## 5.3 Shortcomings of the Current Filtering Method
The current filtering method described in clause [5.1](#51-current-method-of-filtering-in-onem2m) has limitations that prevents to run more sophisticated RETRIEVE and DISCOVERY requests in the CSE. This means that an AE that needs more elaborate filtering capabilities needs to implement them itself, which in case of a restricted device is not always easy or even possible.
The current filtering method described in clause [5.1](#51-current-method-of-filtering-in-onem2m) has limitations that prevents to run more sophisticated RETRIEVE and DISCOVERY requests in the CSE. This means that an AE that needs more elaborate filtering capabilities has to implement them itself, which in case of a restricted device is not always easy or even possible.
To summarise the limitations:
...
...
@@ -257,9 +227,9 @@ An application entity wants to retrieve only those <container> resources wher
- Filter on the values of attributes in child resources.
### 6.2.3 Search for a Tag in *label* or Where Different Attributes are Over a Specific Values
### 6.2.3 Search for a Tag in *label* OR Where Different Attributes are Larger Than a Specific Values
An application entity needs to retrieve resources that have specific a tag present in a *label* attribute, OR those <flexContainer> specialisations where custom attributes *a* AND *b* are BOTH over specific values.
An application entity needs to retrieve resources that have specific a tag present in a *label* attribute, OR those <flexContainer> specialisations where custom attributes *a* AND *b* are BOTH larger than specific values.
**Requirement(s)**
...
...
@@ -422,64 +392,56 @@ However, they are tailored to the specific data formats they are designed to sup
## 7.3.1 Enhancing the Existing *Filter Criteria* Method
The main limitation of the current filtering in oneM2M is the lack of recursivity, which forbids to express complex requests,
and the fact that only the OR logical operator can be used between matching conditions of the same type (i.e. attributes).
The main limitation of the current filtering in oneM2M is the lack of recursivity, which forbids to express complex requests, and the fact that only the OR logical operator can be used between matching conditions of the same type (i.e. attributes).
#### 7.3.1.1 Filter Groups
This first proposal therefore focuses on adding a new *filterGroup (fg)* matching condition that allows nesting expressions and enforces operations
pointed to by the *filterOperation (fo)* tag also between Filter Matching Conditions of the same type.
Making the operation type defined in *filterOperation* tag valid for all the Filter Matching Conditions both makes the syntax more clear/readable
and allows using all AND/OR/XOR/NOT operations also for attributes.
This first proposal therefore focuses on adding a new *filterGroup (fg)* matching condition that allows nesting expressions and enforces operations pointed to by the *filterOperation (fo)* tag also between Filter Matching Conditions of the same type. Making the operation type defined in *filterOperation* tag valid for all the Filter Matching Conditions both makes the syntax more clear/readable and allows using all AND/OR/XOR/NOT operations also for attributes.
With this approach it is not only possible to apply different operations between attribute tags in general but also to use
different operations for each subsequent *filterGroup* when needed.
With this approach it is not only possible to apply different operations between attribute tags in general but also to use different operations for each subsequent *filterGroup* when needed.
Example:
```xml
<filterCriteria>
<fo>2</fo>
<attribute> … </attribute>
<fg>
<fo>1</fo>
<labels> … </labels>
<childLabels> … </childLabels>
</fg>
<fg>
<fo>1</fo>
<attribute> … </attribute>
<fg>
<fo>2</fo>
<labels> … </labels>
<childLabels> … </childLabels>
</fg>
</fg>
</filterCriteria>
<filterCriteria>
<fo>2</fo>
<attribute> … </attribute>
<fg>
<fo>1</fo>
<labels> … </labels>
<childLabels> … </childLabels>
</fg>
<fg>
<fo>1</fo>
<attribute> … </attribute>
<fg>
<fo>2</fo>
<labels> … </labels>
<childLabels> … </childLabels>
</fg>
</fg>
</filterCriteria>
```
This example displays new possibilities to express complex requests:
- Multiple instances of *filterGroup* matching condition allowed.
-*filterGroup* matching conditions can be nested.
- Separate filtering operation defined for each *filterGroup* matching condition.
**Note**
> **Note**
> It is also proposed to remove a quite confusing rule in TS-0004 7.3.3.17.0, saying that:
> "If multiple matching conditions of the same type (i.e. same condition tag) are present in the Filter Criteria parameter, these shall be combined by applying logical OR operation".
It is also proposed to remove a quite confusing rule in TS-0004 7.3.3.17.0, saying that:
“If multiple matching conditions of the same type (i.e. same condition tag) are present in the Filter Criteria parameter,
these shall be combined by applying logical OR operation".
With the proposed extension,
- to send a disjunction request with multiple matching conditions of the same type, the *filterOperation* must be explicitely set to OR.
Though this brings a backward incompatibility, this is limited to matching conditions that originally had 0..n multiplicity,
that is: *contentType*, attributes, childAttributes, parentAttributes, *semanticsFilter*.
- it is allowed to use conjonctions (and XOR operations) requests with multiple matching conditions of the same type,
for instance on attributes (e.g. *attr1=val1 and attr2=val2*), which was not possible before.
- to send a disjunction request with multiple matching conditions of the same type, the *filterOperation* must be explicitely set to OR. Though this brings a backward incompatibility, this is limited to matching conditions that originally had 0..n multiplicity, that is: *contentType*, attributes, childAttributes, parentAttributes, *semanticsFilter*.
- it is allowed to use conjonctions (and XOR operations) requests with multiple matching conditions of the same type, for instance on attributes (e.g. *attr1=val1 and attr2=val2*), which was not possible before.
#### 7.3.1.5 Advantages and Disadvantages
- Major increase in semantic expressiveness with minimal syntactic change: the syntax is the same as the existing one, with only one new filter criterium;
basically, it just allows nesting expressions of the existing request language.
- Major increase in semantic expressiveness with minimal syntactic change: the syntax is the same as the existing one, with only one new filter criterium; basically, it just allows nesting expressions of the existing request language.
- Easy to implement, for the same reason.
- A logical *not* operation is added inside the *filterGroup*.
- A logical *NOT* operation is added inside the *filterGroup*.
<br/>
...
...
@@ -492,6 +454,8 @@ for instance on attributes (e.g. *attr1=val1 and attr2=val2*), which was not pos
The input contribution SDS-2022-0014R01 proposes to add a new attribute *advancedQuery* to ***Filter Criteria*** that contains a filter string with a syntax based on S-expressions <ahref="#_ref_i.2">[i.2]</a>. The new attribute, *advancedQuey*, can be used together in combination with the already existing attributes in ***Filter Criteria*** or alone.
It is an extension of the proposal in clause 7.3.1; the *filterGroup* instances are move from a list attribute to S-expressions.
#### 7.3.2.1 S-expressions
S-expressions are a notation that is based on a nested list (or tree) structure. Those lists may contain either none, one or many elements. Each of the elements can either be value or another S-expression. The first element can also be a function which takes the other following elements of the list as input (after recursively evaluating them). S-expressions are the basis for lisp-like languages, like *Common Lisp*, *Scheme*, or *Closure*.
...
...
@@ -584,6 +548,8 @@ Some functionality is needed to support iterations over direct child resources a
- The size of filter queries formulated as S-expressions is also small. This reduces the size of requests.
- The syntax is simple and easy to understand. The operators and functions can be defined specifically for by oneM2M purposes.
<br />
- There is no implementation, framework or library available.
- The specification of the syntax and the operators and functions needs to be maintained by oneM2M.
