Skip to content
Snippets Groups Projects
Commit 8b7a660e authored by Andreas Kraft's avatar Andreas Kraft Committed by Miguel Angel Reina Ortega
Browse files

Continue with query language description and requirements

parent 7f0a3a18
No related branches found
No related tags found
2 merge requests!3SDS-2024-0077R01-TR-0070_0_0_1_baseline,!2SDS-2024-0077-TR-0070_0_0_1_baseline
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
|-|-| |-|-|
|Document Number |TR-0070-V-0.0.1 | |Document Number |TR-0070-V-0.0.1 |
|Document Name: |Enhanced filtering and Queries | |Document Name: |Enhanced filtering and Queries |
|Date: |<mark>2023-06-15</mark> | |Date: |<mark>2023-10-10</mark> |
|Abstract: |This TR provides an overview about the current status of querying and filtering in oneM2M, formulates new use cases, and provides an overview about relevant query languages. | |Abstract: |This TR provides an overview about the current status of querying and filtering in oneM2M, formulates new use cases, and provides an overview about relevant query languages. |
|Template Version: January 2020 (do not modify) | | |Template Version: January 2020 (do not modify) | |
...@@ -45,26 +45,37 @@ NO REPRESENTATION OR WARRANTY IS MADE THAT THE INFORMATION IS TECHNICALLY ACCURA ...@@ -45,26 +45,37 @@ NO REPRESENTATION OR WARRANTY IS MADE THAT THE INFORMATION IS TECHNICALLY ACCURA
# Contents # Contents
[Contents](#contents) [1 Scope](#1-scope)
[1 Scope](#1-scope) [2 References](#2-references)
[2 References](#2-references) &nbsp;&nbsp;&nbsp;&nbsp;[2.1 Normative references](#21-normative-references)
&nbsp;&nbsp;&nbsp;&nbsp;[2.1 Normative references](#21-normative-references) &nbsp;&nbsp;&nbsp;&nbsp;[2.2 Informative references](#22-informative-references)
&nbsp;&nbsp;&nbsp;&nbsp;[2.2 Informative references](#22-informative-references) [3 Definition of terms, symbols and abbreviations](#3-definition-of-terms,-symbols-and-abbreviations)
[3 Definition of terms, symbols and abbreviations](#3-definition-of-terms,-symbols-and-abbreviations) &nbsp;&nbsp;&nbsp;&nbsp;[3.1 Terms](#31-terms)
&nbsp;&nbsp;&nbsp;&nbsp;[3.1 Terms](#31-terms) &nbsp;&nbsp;&nbsp;&nbsp;[3.2 Symbols](#32-symbols)
&nbsp;&nbsp;&nbsp;&nbsp;[3.2 Symbols](#32-symbols) &nbsp;&nbsp;&nbsp;&nbsp;[3.3 Abbreviations](#33-abbreviations)
&nbsp;&nbsp;&nbsp;&nbsp;[3.3 Abbreviations](#33-abbreviations) [4 Conventions](#4-conventions)
[4 Conventions](#4-conventions) [5 Introduction](#5-introduction)
[5 User defined clause(s) from here onwards](#5-user-defined-clause%28s%29-from-here-onwards) &nbsp;&nbsp;&nbsp;&nbsp;[5.1 Current Method of Filtering in oneM2M](#51-current-method-of-filtering-in-onem2m)
&nbsp;&nbsp;&nbsp;&nbsp;[5.1 User defined subdivisions of clause(s) from here onwards](#51-user-defined-subdivisions-of-clause%28s%29-from-here-onwards) &nbsp;&nbsp;&nbsp;&nbsp;[5.2 Semantic Discovery and Query](#52-semantic-discovery-and-query)
[Proforma copyright release text block](#proforma-copyright-release-text-block) &nbsp;&nbsp;&nbsp;&nbsp;[5.3 Shortcomings of the Current Filtering Method](#53-shortcomings-of-the-current-filtering-method)
&nbsp;&nbsp;&nbsp;&nbsp;[Annexes](#annexes) &nbsp;&nbsp;&nbsp;&nbsp;[5.4 Evaluation Modes of &lt;action> and &lt;dependency> Resource Types](#54-evaluation-modes-of-%26ltaction%3E-and-%26ltdependency%3E-resource-types)
[Annex &lt;A>:Title of annex](#annex-%26lta%3E%3Atitle-of-annex) [6 Requirements for IoT & oneM2M Resource Querying and Filtering](#6--requirements-for-iot-%26-onem2m-resource-querying-and-filtering)
[Annex &lt;B>:Title of annex](#annex-%26ltb%3E%3Atitle-of-annex) &nbsp;&nbsp;&nbsp;&nbsp;[6.1 Existing Requirements in TS-0002](#61-existing-requirements-in-ts-0002)
&nbsp;&nbsp;&nbsp;&nbsp;[ First clause of the annex](#----first-clause-of-the-annex) &nbsp;&nbsp;&nbsp;&nbsp;[6.2 New Use Cases](#62-new-use-cases)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[B.1.1 First subdivided clause of the annex](#b11----first-subdivided-clause-of-the-annex) &nbsp;&nbsp;&nbsp;&nbsp;[6.3 Requirements](#63-requirements)
[Annex &lt;y>:Bibliography](#annex-%26lty%3E%3Abibliography) [7 Query Languages](#7-query-languages)
[History](#history) &nbsp;&nbsp;&nbsp;&nbsp;[7.1 Query Languages Comparison](#71-query-languages-comparison)
&nbsp;&nbsp;&nbsp;&nbsp;[7.3 Alternatives to Enhance oneM2M's Filters and Queries](#73-alternatives-to-enhance-onem2m's-filters-and-queries)
&nbsp;&nbsp;&nbsp;&nbsp;[7.3.1 Enhancing the Existing *Filter Criteria* Method](#731-enhancing-the-existing-*filter-criteria*-method)
&nbsp;&nbsp;&nbsp;&nbsp;[7.3.2 Adding a Filter and Query Language](#732-adding-a-filter-and-query-language)
&nbsp;&nbsp;&nbsp;&nbsp;[7.4 Integration with Filter Criteria](#74-integration-with-filter-criteria)
[8 Conclusion](#8-conclusion)
[Annexes](#annexes)
[Annex &lt;A>:Title of annex](#annex-%26lta%3E%3Atitle-of-annex)
&nbsp;&nbsp;&nbsp;&nbsp;[ First clause of the annex](#----first-clause-of-the-annex)
[Annex &lt;y>:Bibliography](#annex-%26lty%3E%3Abibliography)
[History](#history)
# 1 Scope # 1 Scope
This document presents possible requirements for an enhanced filtering and query mechanism as well as a discussion about possible solutions. This document presents possible requirements for an enhanced filtering and query mechanism as well as a discussion about possible solutions.
...@@ -85,6 +96,7 @@ The following referenced documents are necessary for the application of the pres ...@@ -85,6 +96,7 @@ The following referenced documents are necessary for the application of the pres
The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area. The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area.
- <a name="_ref_i.1">[i.1]</a> oneM2M Drafting Rules [https://member.onem2m.org/static_Pages/others/Rules_Pages/oneM2M-Drafting-Rules-V1%202%202.doc](https://member.onem2m.org/static_Pages/others/Rules_Pages/oneM2M-Drafting-Rules-V1%202%202.doc) - <a name="_ref_i.1">[i.1]</a> oneM2M Drafting Rules [https://member.onem2m.org/static_Pages/others/Rules_Pages/oneM2M-Drafting-Rules-V1%202%202.doc](https://member.onem2m.org/static_Pages/others/Rules_Pages/oneM2M-Drafting-Rules-V1%202%202.doc)
- <a name="_ref_i.2">[i.2]</a> Wikipedia entry on "S-expressions" [https://en.wikipedia.org/w/index.php?title=S-expression&oldid=1178440297](https://en.wikipedia.org/w/index.php?title=S-expression&oldid=1178440297)
# 3 Definition of terms, symbols and abbreviations # 3 Definition of terms, symbols and abbreviations
...@@ -92,38 +104,15 @@ The following referenced documents are not necessary for the application of the ...@@ -92,38 +104,15 @@ The following referenced documents are not necessary for the application of the
## 3.1 Terms ## 3.1 Terms
<mark>Clause numbering depends on applicability.</mark> For the purposes of the present document, the following terms and definitions apply:
- <mark>A definition shall not take the form of, or contain, a requirement.</mark>
- <mark>The form of a definition shall be such that it can replace the term in context. Additional information shall be given only in the form of examples or notes (see below).</mark>
- <mark>The terms and definitions shall be presented in alphabetical order.</mark>
For the purposes of the present document, the [following] terms and definitions [given in ... and the following] apply:
<mark>Definition format</mark>
<mark>&lt;defined term>: &lt;definition></mark>
<mark>If a definition is taken from an external source, use the format below where [N] identifies the external document which must be listed in Section 2 References.</mark>
<mark>&lt;defined term>[N]: &lt;definition></mark> <mark>TODO</mark>
<mark>example 1: text used to clarify abstract rules by applying them literally</mark>
> NOTE: This may contain additional information. SPARQL: SPARQL is a semantic query language to retrieve and manipulate RDF data. It is a W3C specification.
## 3.2 Symbols ## 3.2 Symbols
<mark>Clause numbering depends on applicability.</mark> N/A
For the purposes of the present document, the [following] symbols [given in ... and the following] apply:
<mark>Symbol format</mark>
<mark>`&lt;symbol> &lt;Explanation>`</mark>
<mark>`&lt;2nd symbol> &lt;2nd Explanation>`</mark>
<mark>`&lt;3rd symbol> &lt;3rd Explanation>`</mark>
## 3.3 Abbreviations ## 3.3 Abbreviations
...@@ -131,15 +120,19 @@ For the purposes of the present document, the following abbreviations apply: ...@@ -131,15 +120,19 @@ For the purposes of the present document, the following abbreviations apply:
<mark>TODO</mark> <mark>TODO</mark>
CSE:... CSE: Common Services Entity
IMEI: International Mobile Equipment Identity
JSON: JavaScript Object Notation
IMEI: ... RDF: Resource Description Framework
JSON: ... SQL: Structured Query Language
XML: XML: Extensible Markup Language
SPARQL:... XSD: XML Schema Definition
# 4 Conventions # 4 Conventions
...@@ -155,29 +148,31 @@ The current method for providing filters and queries in oneM2M requests is curre ...@@ -155,29 +148,31 @@ The current method for providing filters and queries in oneM2M requests is curre
To summarise its use: To summarise its use:
- Matching conditions are evaluated against resources. When the evaluation is true, the matched resources are part of the matching result. - *Matching conditions* are evaluated against resources. When the evaluation is true, the matched resources are part of the matching result.
- The filter handling conditions provide additional input used to determine the filtering result (e.g. maximum number of resources to be included in the filtering result). - The *filter handling conditions* provide additional input used to determine the filtering result (e.g. the maximum number of resources to be included in the filtering result).
- The filtering result may be composed of one or more resources and determines the target resources for Discovery-based operations. - The *filtering result* may be composed of zero, one or more resources and determines the target resources for Discovery-based operations.
Further details are also provided in TS-0001 [[1](#_ref_1)] , clause 10.2.6 "Discovery-related Procedures". Further details are also provided in TS-0001 [[1](#_ref_1)] , clause 10.2.6 "Discovery-related Procedures".
The *matching conditions* is a set of attributes that specify for some common and universal resource attributes matching (or comparison) a value and the kind of comparison that should be applied if present. For example, the value of the *createdBefore* attribute is compared against the *creationTime* attribute during matching, and it will be matched if a resource's *createdTime* attribute is before the timestamp specified in *createdBefore*. Through the specific nature of the attributes it is possible to compare against the respective attributes in resources not only for equality, but also for smaller, larger, before, after, etc. The *matching conditions* is a set of attributes that specify for some common and universal resource attributes matching (or comparison) a value and the kind of comparison that should be applied if present. For example, the value of the *createdBefore* attribute is compared against the *creationTime* attribute during matching, and it will be matched if a resource's *createdTime* attribute is before the timestamp specified in *createdBefore*. Through the specific nature of the attributes it is possible to compare against the respective attributes in resources not only for equality, but also for smaller, larger, before, after, etc. However, these attribute-specific comparisons are only defined for a limited set of resource attributes.
Besides the pre-defined set of **_FILTER CRITERIA_** attributes it is possible to compare all these attributes attributes, and also the custom attributes in &lt;flexContainer> specialisations, against equality. However, equality is the only means of comparison for most of oneM2M's attributes. Besides the pre-defined set of **_FILTER CRITERIA_** attributes it is possible to compare all these attributes attributes, and also the custom attributes in &lt;flexContainer> specialisations, against equality. However, equality is the only comparison type for most of oneM2M's attributes.
It is possible to add multiple queries and to logical combine them with setting the *filterOperations* **_FILTER CRITERIA_** attribute. This attribute specifies the logical operator that is used for all the given filter expressions. No sub-queries are allowed, eg. to mix different boolean operations. Negation of query results is also not possible. It is possible to add multiple queries and to logical combine them with setting the *filterOperations* **_FILTER CRITERIA_** attribute. This attribute specifies the logical operator that is used for all the given filter expressions. No sub-queries are allowed, eg. to mix different boolean operations. Negation of query results is also not possible.
In addition **_FILTER CRITERIA_** *filter handling* attributes are used to limit the number of filter result, specifying an offset in the filter result, or limit the search depth in the resource tree. In addition **_FILTER CRITERIA_** *filter handling* attributes are used to limit the number of filter result, specifying an offset in the filter result, or limit the search depth in the resource tree.
<mark>TODO: Example</mark>
## 5.2 Semantic Discovery and Query ## 5.2 Semantic Discovery and Query
Both semantic discovery and query are not part of this report because they use a different method for handling queries, but the should be shortly summarised here. Both semantic discovery and query are not part of this report because they use a different method for handling queries, but they still should be shortly summarised here because they are an example of an additional query method in oneM2M that work together with the methods provided by the ***Filter Criteria*** request attribute.
Semantic queries are different from the filter method described in clause [5.1](#51-current-method-of-filtering-in-onem2m). The filtering methods described in that clause are part of the request handling. A result set of resources or references is created or filtered afterwards according to the given configuration in the **_FILTER CRITERIA_** request attribute. Semantic queries are different from the filter method described in clause [5.1](#51-current-method-of-filtering-in-onem2m). The filtering methods described in that clause are part of the request handling. A result set of resources or references is created or filtered afterwards according to the given configuration in the **_FILTER CRITERIA_** request attribute.
For semantic discovery and query two different methods are defined that use semantic service components to compute the result set according not only by just comparing queries and values, but by deriving explicit and implicit information from the semantic information that are provided together with the resources as well as from external information that are semantically referenced. For semantic discovery and query two different methods are defined that use semantic service components to compute the result set according not only by just comparing queries and values, but by deriving explicit and implicit information from the semantic information that are provided together with the resources as well as from external information that are semantically referenced.
Resources may have child resources of type &lt;semanticDescriptor>, which contain additional semantic information that may describe resource resp. the data contained in that resource further. For example, the data from a temperature sensor may be annotated this way to explain that data producing device is a thermostat used indoor in an industrial context, and that the data is in degree Celsius. Resources may have child resources of type &lt;semanticDescriptor>, which contain additional semantic information that may describe a resource respective the data contained in that resource further. For example, the data from a temperature sensor may be annotated this way to explain that data producing device is a thermostat used indoor in an industrial context, and that the data is in degree Celsius. The same semantic method can be used to semantically describe the device itself.
The differences between semantic discovery and semantic query operations are: The differences between semantic discovery and semantic query operations are:
...@@ -185,24 +180,26 @@ The differences between semantic discovery and semantic query operations are: ...@@ -185,24 +180,26 @@ The differences between semantic discovery and semantic query operations are:
- A *semantic query* is used to send and process a semantic expression by the semantic service component of the CSE. It may use all the available semantic information from the resources in the resource tree. The result, however, is not a set of resources or resource references, but a semantic result string (usually RDF triples). - A *semantic query* is used to send and process a semantic expression by the semantic service component of the CSE. It may use all the available semantic information from the resources in the resource tree. The result, however, is not a set of resources or resource references, but a semantic result string (usually RDF triples).
The query in both cases is a SPARQL query. The query format used in both cases is SPARQL.
## 5.3 Shortcomings of the Current Filtering Method ## 5.3 Shortcomings of the Current Filtering Method
The current filtering method described in clause [5.1](#51-current-method-of-filtering-in-onem2m) has limitations that is preventing to execute more sophisticated RETRIEVE and DISCOVERY requests in the CSE. This means that an AE that want to make use of this feature needs to implement them itself, which in case of a restricted device is not always easy or even possible. The current filtering method described in clause [5.1](#51-current-method-of-filtering-in-onem2m) has limitations that prevents to run more sophisticated RETRIEVE and DISCOVERY requests in the CSE. This means that an AE that needs more elaborate filtering capabilities needs to implement them itself, which in case of a restricted device is not always easy or even possible.
To summarise the limitations: To summarise the limitations:
- Comparison operations such as "greater", "less", "earlier", "later" are defined for only a limited set of common and universal attributes. - Comparison operations such as "greater", "less", "earlier", "later" are defined for only a limited set of common and universal attributes.
- Other attributes, including &lt;flexContainer> custom attributes can only be compared for equality. - Other resource attributes that are not part of this predefined set, including &lt;flexContainer> custom attributes, can only be compared for equality.
- Filtering of individual attributes in complex attributes is not possible. - Filtering of attributes that are part of complex attributes is not possible.
- Filtering on elements of a list is not possible. - Filtering on elements of a list or sub-strings in a string is not possible.
- Filter expressions that contain multiple attributes can only be combined via "AND", "OR", or "XOR" logical operators. - Filter expressions that contain multiple attributes can only be combined via "AND", "OR", or "XOR" logical operators.
- Filter expressions cannot be negated (logical operator "NOT") - Filter expressions cannot be negated (logical operator "NOT")
## 5.4 Shortcomings of Evaluation Modes of &lt;action> and &lt;dependency> Resource Types ## 5.4 Evaluation Criteria of &lt;action>, &lt;dependency>, and &lt;state> Resource Types
In addition to the queries and filters provided in requests some resources also use a similar method to perform their tasks. The &lt;action> and &lt;dependency> resources provide a simple method to react on changes in other resources without notifying an Application Entity to perform the necessary requests itself. In the resource type &lt;state> the *evalCriteria* attribute is used to control state transitions.
In addition to the queries and filters provided in requests some resources also use a similar method to perform their tasks. The &lt;action> and &lt;dependency> resources provide a simple method to react on changes in other resources without going through an application entity to perform the necessary requests. Both resource types have attributes that contain an evaluation criteria that is used for the comparison as soon as an update happens to a target resource. This is sufficient for simple scenarios, more sophisticated scenarios, for example evaluating multiple or complex attributes is not possible. These resource types have attributes that contain an evaluation criteria that is used for the comparison as soon as an update happens to a target resource. This is sufficient for simple scenarios, such as switching on and off a light depending on the value of an updated *content* attribute in a &lt;container> resource, but more sophisticated scenarios, for example evaluating multiple or complex attributes, are not possible.
To summarise the limitations: To summarise the limitations:
...@@ -233,27 +230,37 @@ The first requirement implies a technical filter or query while the other requir ...@@ -233,27 +230,37 @@ The first requirement implies a technical filter or query while the other requir
## 6.2 New Use Cases ## 6.2 New Use Cases
The following is a collection of new use cases that are interesting for IoT application as well as for additional functionality for common services within the CSE. The following is a list of new use cases that are interesting for IoT application as well as for additional functionality for common services within the CSE.
### 6.2.1 Searching for an IMEI in *label* ### 6.2.1 Searching for an IMEI in the *label* Attribute
An application entity needs to store extra meta tags in a resource, e.g. the IMEI number of the hosting devices, and other tags. For this, the application entity uses the *label* attribute. Another application entity that wants to retrieve resources by the IMEI number wants to query for these resources by only the single tag in the *label* attribute. An application entity needs to store extra meta tags in a resource, e.g. the IMEI number of the hosting devices, and other tags. For this, the application entity uses the *label* attribute. Another application entity that wants to retrieve resources by the IMEI number wants to query for these resources by only the single tag in the *label* attribute.
**Requirement(s)** **Requirement(s)**
- Filter on a single tag in a list attribute. - Filter on a single tag (string) in list attribute.
### 6.2.2 Monitor Temperature Values
### 6.2.2 Monitor Temperature Values #### 6.2.2.1 Compare Using &lt;flexContainer> Resources
An application entity wants to retrieve only those &lt;flexContainer> [temperature] specialisations where the *currentTemperature* attribute is above a certain value. An application entity wants to retrieve only those [temperature] &lt;flexContainer> specialisations where the *currentTemperature* attribute is above a certain value.
**Requirement(s)** **Requirement(s)**
- Filter on a custom attribute that is greater than a comparison value. - Filter on a custom attribute that is greater than a comparison value.
### 6.2.3 Search for a Tag in *label* or Where Multiple Attributes are Over a Specific Values #### 6.2.2.2 Compare Using &lt;container> and &lt;contentInstance> Resources
An application entity needs to retrieve resources that have specific tag added to a *label* attribute, OR which attributes *a* and *b* are BOTH over specific values. An application entity wants to retrieve only those &lt;container> resources where any of the child &lt;contentInstance> resources contains a temperature above a certain value.
**Requirement(s)**
- Filter on the values of attributes in child resources.
### 6.2.3 Search for a Tag in *label* or Where Different Attributes are Over a Specific Values
An application entity needs to retrieve resources that have specific a tag present in a *label* attribute, OR those &lt;flexContainer> specialisations where custom attributes *a* AND *b* are BOTH over specific values.
**Requirement(s)** **Requirement(s)**
...@@ -269,13 +276,15 @@ An application entity needs to retrieve all the &lt;subscription> resources that ...@@ -269,13 +276,15 @@ An application entity needs to retrieve all the &lt;subscription> resources that
- Filter on an attribute that is part of a complex type (child-attribute of another attribute). - Filter on an attribute that is part of a complex type (child-attribute of another attribute).
### 6.2.5 Reacting on Multiple Attribute Change Using &lt;action> Resources ### 6.2.5 Reacting on Multiple Attribute Changes for &lt;action> Resources
An application entity wants to install an &lt;action> resource that monitors a flexContainer> specialisation with attributes *a* and *b*. If both attributes in that resource rises above a certain value, the &lt;action> resource shall assign a value to an attribute in another resource.
An application entity wants to install an &lt;action> resource that monitors a flexContainer> specialisation with attributes *a* and *b*. If both attributes in that resource rises beyond a certain value, the &lt;action> resource shall set an attribute in another resource to a value. Similar requirements can be deducted for use of the &lt;dependency> and &lt;state> resource types.
**Requirement(s)** **Requirement(s)**
- Be able to evaluate one or more attributes in one expression. - Be able to evaluate one or more attributes in one evaluation criteria.
- Be able to evaluate complex type attributes. - Be able to evaluate complex type attributes.
- Support the same comparison operators as in other queries (e.g. "in") - Support the same comparison operators as in other queries (e.g. "in")
...@@ -283,67 +292,263 @@ An application entity wants to install an &lt;action> resource that monitors a f ...@@ -283,67 +292,263 @@ An application entity wants to install an &lt;action> resource that monitors a f
--- ---
<mark>TODO: Use case for parent- or child-resource attributes</mark>mark <mark>TODO: Use case for parent- or child-resource attributes</mark>
## 6.3 Requirements ## 6.3 Requirements
The following requirements are derived from the comparison above. In addition, some non-function requirements are formulated. The following requirements are derived from the comparison above. In addition, some non-functional requirements are formulated.
### 6.3.1 Functional Requirements ### 6.3.1 Functional Requirements
#### Compare Custom and Resource Specific Attributes Using Comparison Operators Other than only Equality - Being able to compare custom and resource specific attributes
- Being able to use comparison operators other than only equality, such as lesser and greater than
- Support other operators such as "in" for lists and strings
- Provide sub-queries
- Being able to mix boolean operations for individual or sub-queries
- Support the Same Functionality for &lt;action> and &lt;dependency> Resources - Allow to reference resource's parent and child resources and user their attributes in comparisons as well.
- Provide the same functionality for &lt;action> and &lt;dependency> Resources
- <mark>TBC</mark>
### 6.3.2 Non-Functional Requirements ### 6.3.2 Non-Functional Requirements
- Size - Ease-of-use
- Simplicity
- Size of implementation
- Be backward compatible with the existing ***Filter Criteria***, and ideally can be combined
- <mark>TBC</mark>
# 7 Query Languages
==TODO== This section provides an overview about some existing query languages.
# 7 Query Languages Comparisons ## 7.1 Query Languages Comparison
Data query languages are syntaxes designed specifically for querying and retrieving data from databases or data repositories. These query languages provide a simplified way to interact with data, making it easier for applications to retrieve the information they need. Here is a short summary of relevant data query languages: Data query languages are syntaxes designed specifically for querying and retrieving data from databases or data repositories. These query languages provide a way to interact with data, making it easier for applications to retrieve the information they need. Here is a short summary of relevant data query languages:
1. **SQL (Structured Query Language)**: SQL is the most widely used query language for relational databases. It allows users to retrieve, manipulate, and manage data stored in tables. SQL uses a declarative approach, where applications specify what data they want to retrieve rather than how to retrieve it. SQL supports various operations like SELECT, INSERT, UPDATE, and DELETE, enabling users to perform complex queries and aggregations on structured data. Relevant for filtering is the WHERE clause where filter expressions resp. query statements can be formulated. 1. **SQL (Structured Query Language)**: SQL is the most widely used query language for relational databases. It allows users to retrieve, manipulate, and manage data stored in tables. SQL uses a declarative approach, where applications specify what data they want to retrieve rather than how to retrieve it. SQL supports various operations like SELECT, INSERT, UPDATE, and DELETE, enabling users to perform complex queries and aggregations on structured data. Relevant for filtering is the WHERE clause where filter expressions respective query statements can be formulated.
2. **XQuery**: XQuery is a query language specifically designed for querying XML (eXtensible Markup Language) data. It allows users to extract and manipulate data from XML documents using a variety of query and transformation capabilities. XQuery supports XPath, a language for navigating XML structures, making it suitable for querying and transforming complex XML data. ==examples== 2. **XQuery**: XQuery is a query language specifically designed for querying XML (eXtensible Markup Language) data. It allows users to extract and manipulate data from XML documents using a variety of query and transformation capabilities. XQuery supports XPath, a language for navigating XML structures, making it suitable for querying and transforming complex XML data.
3. **JSON Query** (also known as JSONPath) is a syntax used to query and extract specific data from JSON documents provides a way to navigate through the hierarchical structure of JSON data and select elements based on specific criteria. It uses a path-like syntax to traverse the JSON structure and select elements or values. 3. **JSON Query** (also known as JSONPath) is a syntax used to query and extract specific data from JSON documents provides a way to navigate through the hierarchical structure of JSON data and select elements based on specific criteria. It uses a path-like syntax to traverse the JSON structure and select elements or values.
### 7.2 Query Languages Examples
These simple data query languages provide efficient and straightforward methods for retrieving and manipulating data from databases or data sources. They offer different syntaxes and capabilities tailored to specific data formats and use cases, simplifying data retrieval and empowering users to work with data more effectively. > 📝 Could also be moved to annex
The following examples show queries for the query languages introduced before.
#### 7.2.1 SQL
# Annexes The following example queries data records from an addresses table filtered by a specific street name.
## Annex A: Query Examples ```sql
SELECT *
FROM addresses
WHERE street_name = 'Main Street';
```
- SQL The following example queries data records from an addresses table filtered by a specific street name and street number.
- XPATH
- JSONPath ```sql
SELECT *
FROM addresses
WHERE street_name = 'Main Street' AND street_number < 10;
```
The following example queries structured data records from an addresses table by a specific street name.
```sql
SELECT address.street
FROM addresses
WHERE addresses.address.street_number < 10;
```
#### 7.2.2 XQuery
In general, XML and XQuery support organization of and access to complex structured data.
The following example queries elements from an XML document filtered by a specific street name.
```XQuery
for $address in /addresses/address
where $address/street_name = 'Main Street'
return $address
```
The following example queries elements from an XML document filtered by a specific street name and street number.
```xquery
for $address in /addresses/address
where $address/street_name = 'Main Street' and $address/street_number < 10
return $address
```
#### 7.2.3 JSON Query
In general, JSON and JSON Query support organization of and access to complex structured data.
The following example queries elements
```json
$.address[?(@.streetName == "Main Street")].streetName
```
```json
$.address[?(@.streetNumber < 10 && @.streetName == "Main Street")].streetName
```
## 7.3 Alternatives to Improve oneM2M's Filters and Queries
The data query languages in section <mark>7.1</mark> provide efficient and straightforward methods for retrieving and manipulating data from databases or data sources. They offer different syntaxes and support the specific data formats and use cases and simplifying data retrieval.
However, they are tailored to the specific data formats they are designed to support. oneM2M uses XSD, the XML Schema Definition language, to formally define its specification's artefacts (for example the resources and request structures). It would seem that *XQuery* would be a natural choice for a query language in oneM2M. However, it shares some drawbacks with the other presented query languages:
- The presented query languages are more suited for the environments for which they have been designed. oneM2M defines its own data model which is then mapped to either XML, JSON or CBOR when exchanging data with an Application Entity or another CSE during a request/response exchange.
- Though CSE implementation may choose to support any of these encoding formats, they usually only support one or two of them. Adding any of the frameworks to an implementation would probably increase the size of an implementation considerably. This would especially be the case for smaller implementations that would run as an ASN on a constraint device. Those implementations may choose only to support one content encoding (XML, JSON or cbor). Adding a query frameworks that originates from any of the the other, unsupported encodings would add even more size to a deployment.
## 7.3.1 Enhancing the Existing *Filter Criteria* Method
<mark>Cyrille (Orange) contributes</mark>
## 7.3.2 Defining a Simplified New Filter and Query Language
The input contribution SDS-2022-0014R01 proposes to add a new attribute *advancedQuery* to ***Filter Criteria*** that contains a filter string with a syntax based on S-expressions <a href="#_ref_i.2">[i.2]</a>. The new attribute, *advancedQuey*, can be used together in combination with the already existing attributes in ***Filter Criteria*** or alone.
#### 7.3.2.1 S-expressions
S-expressions are a notation that is based on a nested list (or tree) structure. Those lists may contain either none, one or many elements. Each of the elements can either be value or another S-expression. The first element can also be a function which takes the other following elements of the list as input (after recursively evaluating them). S-expressions are the basis for lisp-like languages, like *Common Lisp*, *Scheme*, or *Closure*.
Examples:
- `(1 2 3)` : This S-expression is a list with the three values 1, 2, 3
- `(== 1 2)` : This S-expression evaluates the function `==` with the parameters `1` and `2`. It evaluates to `false`
- `(| (== 1 2) (< 3 4))` : This S-expression evaluates the function `|` (or) with the result of the two S-Expressions `(== 1 2)` (that evaluates to *false*) and `(< 3 4` (that evaluates to *true*). The whole S-expression evaluates to `true`.
The advantage of S-expressions are that parsers and interpreters are fairly simple to implement in any programming language, and that implementations have a small footprint.
#### 7.3.2.2 Supported Comparisons
Since S-expressions are evaluated recursively no loop or expression functionalities need to be supported. The number of operations can be limited to a small set of logical and comparison operations. The following list can be extended if needed.
The S-expression for queries and filters in oneM2M has at least one operator followed by an attribute name or an S-expression, optionally followed by one or more values or S-expressions. Attribute names are replaced by their respective values from the evaluated resources during a query:
```(<operator> <attribute name or S-expression> [<value or S-expression>]*)```
**Table 7.3.2.2-1: Supported Logical Operators**
| Operator | Description | Examples |
| -------- | ------------------------------------------------------------ | --------------------------------- |
| & | Logical *and*. All included expressions must evaluate to *true* for the expression to evaluate to *true*, otherwise the expression evaluates to *false*. | `(& (== ty 1) (in lbl "aLabel"))` |
| \| | Logical *or*. At least one of the included expressions must evaluate to *true* for the expression to evaluate to *true*, otherwise the expression evaluates to *false*. | `(\| (== ty 1) (== ty 2))` |
| ! | Logical *not*. Only one included expression is allowed, which boolean value is negated as a result of the evaluation. | `(! (== ty 1))` |
**Table 7.3.2.2-2: Supported comparison Operators**
| Operator | Description | Examples |
| -------- | ------------------------------------------------------------ | --------------- |
| == | Equality comparison. All included and evaluated expressions are compared to each other. <br />Only if all included expressions are equal to each other the result of the comparison is *true*, otherwise the expression evaluates to *false*. | `(== ty 1)` |
| != | Inequality comparison. All included and evaluated expressions are compared to each other. <br />Only if all included expressions are not equal to each other the result of the comparison is *true*, otherwise the expression evaluates to *false*. | `(!= ty 1)` |
| < | Less-than comparison. All included and evaluated expressions are compared to each other in order.<br />Only if all included expressions are monotonically increasing the result of the comparison is *true*, otherwise the expression evaluates to *false*. | `(< 1 2 3)` |
| <= | Less-than-or-equal comparison. All included and evaluated expressions are compared to each other in order. <br />Only if all included expressions are less than or equal to the value of their right expression the result of the comparison is *true*, otherwise the expression evaluates to *false*. | `(<= 1 2 2 3)` |
| > | Greater-than comparison. All included and evaluated expressions are compared to each other in order.<br />Only if all included expressions are monotonically decreasing the result of the comparison is *true*, otherwise the expression evaluates to *false*. | `(> 3 2 1)` |
| >= | Greater-than-or-equal comparison. All included and evaluated expressions are compared to each other in order. <br />Only if all included expressions are greater than or equal to the value of their right expression the result of the comparison is *true*, otherwise the expression evaluates to *false*. | `(>= 3 2 2 1)` |
| in | Contains comparison. The first expression must evaluate to a string or list. All remaining expressions must be contained in that string or list for the expression to evaluate to *true*, , otherwise the expression evaluates to *false*. | `(in (1 2 ) 2)` |
The following is a list of operators or functions for further extensions that may be included later if needed:
- startsWith, endsWith: compare only the start or beginning of a string
- regular expression matches
- case-insensitive comparisons for strings
- presence: an attribute is present or not, regardless of its value
#### 7.3.2.3 Attribute Paths
To support complex attributes and also parent-child relationships for queries the following syntax options for attribute paths are proposed:
**Table 7.3.2.3-1: Attribute paths syntax**
| Syntax | Description | Example |
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| `attribute` | A single top-level attribute of an evaluate resource | `rn`: ResourceName |
| attribute.sub-attribute<br />attribute.sub-attribute.sub-sub-attribute<br />... | A single attribute that is part of a complex structure. It can be further down the path of complex attributes | `loc.typ`: type of m2m:geoCoordinates |
| attribute{}sub-attribute<br />attribute.sub-attribute{}sub-sub-attribute<br />attribute{}sub-attribute{}sub-sub-attribute<br />... | A single attribute that is a member of a list attribute. During evaluation, all the members of a list are evaluated.<br />For a successful comparison of list attributes, only one occurrence in a list needs to successfully fulfil the comparison evaluation. | `pc.acr{}acod{}chty`: The *chty* attribute in *accessControlObjectDetails*. |
| cattr."attribute path" | Apply an attribute path to all the direct child-resources of the currently tested resource. All the direct child-resources are | |
| ^^."attribute path" | Apply an attribute path to the parent resource of a currently tested resource. | `^^.lbl`: The *labels*** attribute of the parent resource. |
> **Note**
>
> It is common in lisp-like languages to use the `-` character in function and attribute names.
If an attribute is not present in a resource, for example in case of an optional attribute, the expression returns `NULL`, which for a boolean expression evaluates to `False`.
#### 7.3.2.4 Functions and Iterators
Some functionality is needed to support iterations over direct child resources and comparing their attributes against a comparison. The following table lists those functions.
**Table 7.3.2.4-1: Functions and iterators**
| Function | Description | Example |
| ------------ | ------------------------------------------------------------ | --------------------------------------- |
| any-child | Iterate over every direct child resource of a currently tested resource and individually evaluate the argument S-expression. The function will evaluate to `true` if the S-expression evaluates to `true` for any of the direct child resources of the currently tested resource. This function can be applied recursively to include sub-child resources as well. | `(any-child (& (== ty 4) (> con "40"))` : Test if any &lt;contentInstance> has a value that is greater than "40". |
| all-children | Iterate over every direct child resource of a currently tested resource and individually evaluate the argument S-expression. The function will evaluate to `true` if the S-expresson evaluates to `true` for all the direct child resources of the currently tested resource. This function can be applied recursively to include sub-child resources as well. | `(all-children (== ty 3))` : Test if all the child resources are of type &lt;container>. |
#### 7.3.2.5 Advantages and Disadvantages
- An implementation for an S-expression parser and interpreter could be fairly small and would fit in smaller ASN and MN CSE implementations.
- The size of filter queries formulated as S-expressions is also small. This reduces the size of requests.
- The syntax is simple and easy to understand. The operators and functions can be defined specifically for by oneM2M purposes.
- There is no implementation, framework or library available.
- The specification of the syntax and the operators and functions needs to be maintained by oneM2M.
# 8 Conclusion
<mark>TODO</mark>
# Annexes
&lt;Text>
<mark>&lt;PAGE BREAK></mark>
# Annex &lt;B>:<br />Title of annex # Annex &lt;A>:<br />Title of annex
&lt;Text> &lt;Text>
## <!--B.1--> First clause of the annex ## <!--A.1--> First clause of the annex
&lt;Text> &lt;Text>
### B.1.1 First subdivided clause of the annex ### A.1.1 First subdivided clause of the annex
&lt;Text> &lt;Text>
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment