Smart2B T3.1 Lead by ODINS

Issue report form

Please, before reporting your issue, make sure you have reviewed the following sections of this page, as they may contain information about it and potentially how to solve, or the ongoing process in solving it.

Afterwards, if your issue has not been addressed in the following sections, please copy the following issue report form into an email, fill it out and send it to jsanchez@odins.es

Email subject: [Smart2B] Issue Report - <topic>

Reported by:

Email:

Tasks involved/Partners involved:

Broker Entities in particular involved:

Severity (select from Minor, Moderate, Major and Critical):

Summary:

Description:

Error messages obtained:

Requests that produce the errors (so that we can reproduce the issue):

Time interval (UTC time) during which these errors have occurred:

We will try to resolve your issue as soon as possible. Thank you very much and sorry for the inconvenience.

Ongoing Issues known by the ODINS team

WP5 app is having issues in order to plot and get all the data that must be shown in the browser - https://wiki.odins.es/public/smart2b/home#wp5_integration_ongoing_issues

Entity Info Spreadsheet

Always make sure that you are using the latest release, an announce is emailed to the partners participating in the WP2345 Alignment meeting and also a copy is placed in the Sharepoint folder.

Please DO NOT place copies of the spreadsheet yourself in the Sharepoint or circulate it among partners, only ODINS is in charge of releasing the spreadsheet and all the management is done bilaterally.

The Entity Info Spreadsheet is under WP3 / Entity Information Spreadsheet

Authorization Access to smart2b.odins.es resources

Each and every partner should have different credentials to access the Smart2B Platform. These credentials are mandatory and required for each request.

The access policy is bottom down. I.e. initially partners have access to nothing and are granted permissions due to request and only if it makes sense.

Here are the current permissions allowed:

The following credentials have READ access to ALL entities of ALL TYPES in the NGSI-LD Broker and Storage component.

certh@smart2b.com

edpsel@smart2b.com

enerbrain@smart2b.com

fcid@smart2b.com

nordomatic@smart2b.com

xtel@smart2b.com

vito@smart2b.com

rwth@smart2b.com

tug@smart2b.com

For any entity OF ANY TYPE that start with the pattern SCML-.* these accounts have WRITE permissions.

edpsel@smart2b.com

xtel@smart2b.com

For any entity OF ANY TYPE that start with the pattern AirBnB-.* these accounts have WRITE permissions.

edpsel@smart2b.com

fcid@smart2b.com

xtel@smart2b.com

For any entity OF ANY TYPE that start with the pattern AOC-.* these accounts have WRITE permissions.

enerbrain@smart2b.com

For any entity OF ANY TYPE that start with the pattern ABL-.* these accounts have WRITE permissions.

nordomatic@smart2b.com

edpsel@smart2b.com

For any entity OF ANY TYPE that start with the pattern ARA-.* these accounts have WRITE permissions.

nordomatic@smart2b.com

edpsel@smart2b.com

xtel@smart2b.com

For any entity OF ANY TYPE that start with the pattern CJC-.* these accounts have WRITE permissions.

enerbrain@smart2b.com

For all the entities of Type DeviceMeasurement the following accounts have WRITE Access:

certh@smart2b.com

NOTE this is due to how the T3.2 components led by CERTH (Data Filtering, Semmantic Annotation) work.

For all the entities of Type Device the following accounts have WRITE Access:

enerbrain@smart2b.com

NOTE this is due to how the T3.3 components led by EB (Translator) work.

Disaggregation of SCML entities

The SCML entities were disaggregated by petition of Miguel Brito in the following form during October 2023.

Please, always check that you're working with the most up to date version of the spreadsheet since those entities changed names:

At the moment of writing this message, the one currently is the v2.1 of the spreadsheet.

https://edponcloud.sharepoint.com/:f:/r/teams/O365_H2020Smart2B/Shared%20Documents/General/WP3%20-%20Development%20of%20SMART2B%20Platform%20%26%20APIs/Entity%20Information%20Spreadsheet?csf=1&web=1&e=TuC7UB

------------------------------------------------------------------------------------------
AirBnB-Mezaninne: (First Left:58, First Right:54, Ground Right:35)
AirBnB-Mezaninne_FirstFloor-Right --> AirBnB-MezaninneFirstRight_FirstFloor-Right
AirBnB-Mezaninne_FirstFloor-Left --> AirBnB-MezaninneFirstLeft_FirstFloor-Left
AirBnB-Mezaninne_GroundFloor-Right --> AirBnB-MezaninneGroundRight_GroundFloor-Right
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
SCML-LargoOttolini
SCML-LargoOttolini_FirstFloor-Right --> SCML-LargoOttoliniFirst_FirstFloor-Right
SCML-LargoOttolini_GroundFloor-Right --> SCML-LargoOttoliniGround_GroundFloor-Right
SCML-LargoOttolini_SecondFloor-Left --> SCML-LargoOttoliniSecond_SecondFloor-Left
------------------------------------------------------------------------------------------
SCML-MarquesadeAlorna: 132 (ground:63, second:69)
SCML-MarquesadeAlorna_GroundFloor-Right --> SCML-MarquesadeAlornaGround_GroundFloor-Right
SCML-MarquesadeAlorna_SecondFloor-Right --> SCML-MarquesadeAlornaSecond_SecondFloor-Right
------------------------------------------------------------------------------------------
SCML-RuadaVinha:240 (ground:78, first:78, second: 84)
SCML-RuadaVinha_FirstFloor --> SCML-RuadaVinhaFirst_FirstFloor
SCML-RuadaVinha_SecondFloor --> SCML-RuadaVinhaSecond_SecondFloor
SCML-RuadaVinha_ThirdFloor --> SCML-RuadaVinhaThird_ThirdFloor
------------------------------------------------------------------------------------------

Integrating Partner FAQ

What is said here must be complemented with the contents of Smart2B Deliverable D3.1 Section 8, where all the steps to perform operations against the NGSI-LD Broker and Storage component are thoroughly documented.

Each integrating partner is supposed to only perform PATCH operations against the DeviceMeasurement entities that are relevant to their pilots (see list above). If an integrating partner requests an authorization token to entities that are not within their administrative domain - I.e. belong to a different pilot - the request to generate an authorization request will return an error.

For the most part, integrating partners (I.e. XTEL, EDP SEL, EB, Nordomatic) might forget about performing either GET or PATCH operations against Building and Zone entities. Those are required for other WP4 services but not really relevant otherwise for building equipment integration.

Due to several circumstances, The name of the entities in the broker may change. Although we, as a consortium, try our best to avoid this matter, it is unavoidable. Hence, Make sure that you're using the entities from the latest release of the Entity Info Collection Spreadsheet, located in the WP3 Folder in the Sharepoint Folder.

Otherwise, if you employ names not available in the Broker, the behaviour is undefined, and most likely not error-prone.

Make sure that when you request the authorization token, the URL does contain the right pilot prefix:

E.g.

If a partner wishes to perform a PATCH request against the following entity:

ABL-RetirementHome_ThirdFloor-NA-BuildingP-Z3050Z1_Nordomatic-BMS-NA-RT01_Temperature

Note that the entity starts with ABL-, hence, the right prefix to request an authorization token would be ABL-.* as shown below:

{"token": "{{idmToken}}","ac": "PATCH", "de": "https://smart2b.odins.es:1027", "re": "/ngsi-ld/v1/entities/urn:ngsi-ld:DeviceMeasurement:ABL-.*/attrs"}

Note the ABL-.* at the end of the URL. As a consequence, this authorization token will be valid only to perform the PATCH operation to those entities. If the developer wishes to perform PATCH operations against entities that have a different prefix (e.g. ARA-), then the authorization token must be adjusted.

Smart2B platform resource monitoring processes

Currently we have a series of processes that are executed automatically with certain periodicity and that are responsible for monitoring the status of the different resources of the machine where the Smart2B services are hosted.

These processes notify us by email in case that the percentage of disk, RAM or CPU usage of the machine exceeds certain values, so that we can act as soon as possible and prevent this from becoming a performance problem for Smart2B services.

In addition, we have another process that is responsible for periodically recording in a log the percentage of disk, RAM and CPU usage of the machine, so that we can know the status of the machine at all times and thus have more information in order to deal with possible incidents related to the operation of Smart2B services.

Smart2B services monitoring and restart processes

We have a series of processes that run automatically and are responsible for monitoring the correct functioning of the Smart2B platform services, and also for restarting those services whose functioning is not correct.

Scheduled restart of security components

We have an automatic process that runs every 4 hours and restarts the security components of the Smart2B platform: Keyrock, Capability Manager, PEP-Proxy, Historical PEP-Proxy and XACML.

The reason for these scheduled reboots is that they help us automatically refresh the security certificates (which even if we renew them on the Smart2B machine, they are not updated until the security components are restarted), and we also discovered that these reboots are good for prevent specific blockages of security components.

Monitoring of security components

We have other automatic processes that run every minute and check that the APIs of the security components respond correctly, leaving a timeout of 10 seconds. If after this time the API does not respond, or if it does not respond correctly, these automatic processes will restart the corresponding components and will record these restarts in a log file.

The scheduled reboots mentioned above are not recorded in this log file.

Scheduled restart of the Elasticsearch historical service

We have an automatic process that runs daily and restarts the Elasticsearch historical service of the Smart2B platform.

Orion NGSI-LD broker monitoring

We also have an automatic process that runs every minute and checks if the Orion NGSI-LD broker service of the Smart2B platform is stopped or restarting, and if so, recreates the container of said service (without affecting the database).

WP5 Integration Ongoing issues

It takes a lot of time to get the data for the WP5 app whenever it queries the QuantumLeap + CrateDB for data to plot.

This is very likely related with the complete chain that provides data to the QuantumLeap & CrateBD components in T3.2

Aliki posts data each 60 minutes back to the QuantumLeap.

[X] (ODINS) review the last emails with the exchanges to catch-up.

[ ] (Jesús ODINS) Retrieve the queries that WP5 Manuel Fonseca FC.ID is using to query data from the platform

[X] (Sergio ODINS) Alternatively, use the credentials provided by FC.ID in order to try the requests interacting with the website to tray to accelerate working in this.

[X] Nuno Matheus (EDP) reports that the platform is not working properly

[X] Assess exactly what components are not working.

[X] (Sergio ODINS) Aliki's email 2023-12-05 gives further information on why there was missing data from the QL.

[ ] OdinS has in place several different service monitoring processes running in the background that did not raise any alarm. Nevertheless, this monitoring processes did not raise any alarm - look into it.

[X] (Sergio ODINS) - briefly document in maybe half a screen the service monitoring processes that we have in the background - what they monitor an the frequency to document the issues.

[ ] No data has been found for November 2023 for:

SCML SantaTeresinha

[X] (Sergio ODINS) Aliki's email 2023-12-05 gives further information on why there was missing data from the QL.

[ ] Assess in which step of the chain the issue might be happening: Sensors - Integratio - Platform - Data Filter - QuantumLeap

[X] (Sergio ODINS) Aliki's email 2023-12-05 gives further information on why there was missing data from the QL.

[X] (Sergio ODINS) Nuno Dionisio (FC.ID) asks what the Index from the requests to the QuantumLeap are.

[ ] (Jesús ODINS) The energy consumption data published against the Broker might be different.

(Jesús ODINS) In some cases it might be a cumulative counter that never stops increasing.

(Jesús ODINS) In other cases, the sensor reports each X period of minutes with the power consumed in that particular period of time.

[ ] (Jesús ODINS) Clarify which cases do we have in the project and set in place a mechanism that distinguishes among them to avoid misinterpretation by the APP.

[ ] (FOR LATER) One of the biggest concerns for FC.ID is that there is a summary panel with all the information about a whole building in one single screen. This summary takes too much time to process since the queries are too slow. Hence, the matter should be reviewed after addressing the previous points so that the queries do not take so long and the summary screen can be shown in an acceptable rate.

[ ] (FOR LATER) NICE TO HAVE - FC.ID could set some kind of cache memory in ephemeral storage for the data that was queried previously in case that the data cannot be retrieved from the platform at an acceptable rate.

Dear Nuno, Aliki, and Jesus,
We (WP5) have been connecting the WebApp with the API to access the data about consumption, temperature, CO2, etc., and we would like to report some situations and make some requests.
## Situations (assuming we are doing our calls correctly)
- Currently, the application takes a lot of time:
- login, which includes getting the list of buildings belonging to the user: 1m23s
- Receiving the data for the Details: 1m45s
- Last Month's details: 2m30s
- Missing data:
- There is only data until the end of October for temperature, CO2, and humidity (See "This Year" in the Time Interval)
- Only consumption information is available for "Today"
Please test it yourself to determine where things are taking too long and if we are doing something wrong during the requests.
- WebApp: https://project-smart2b.lasige.di.fc.ul.pt/
- username/passwd: <REDACTED>
## Requests
- Data sent on each request
- Currently, you are sending the average, maximum, and minimum values for each request (regardless of the type of information being requested)
- However, it should be different for consumption/production and temperature/CO2/humidity
- In the case of consumption/production, the value per hour/day/month should be the overall energy of that period, while for temperature/CO2/humidity it should be the average during the period
- Thus, to simplify, when we request values, you just need to send one value (we will compute the average, maximum, and minimum for the presented chart).
- Case 1: If we ask for the consumption/production in the "Last Month", we need/want to receive the total consumption/production for each day of the month.
- Case 2: If we ask for the temperature/CO2/humidity in the "Last Month", we need/want to receive the average temperature/CO2/humidity for each day of the month.
- In terms of the JSON sent for each request, it can be similar to this:
{
"id": "AOC-ForumComplex_Temperature",
"numValue": [
{
"value": 26.330648104349773,
"value_eur": 0.0,
"observedAt": "2023-10-01T00:00:00.000+00:00"
}
]
}
- Data for the overview screen of a building
- In the overview screen, we show consumption, production, comfort, Status, and well-being data. As it is, to show all the values, we need to perform several requests to the API (which will be even slower). To optimize access to this information, we suggest the creation of a new end-point, which will provide all the information in a single request/JSON.
- The idea is that these values will be computed automatically every hour (or another time interval more suitable) and stored in the platform, so when the WebApp performs a request all the values are already computed and ready to send in a single JSON
- We need the following values for consumption/production:
- Total consumption/production of the current day
- Total consumption/production of the current month
- Total consumption/production of the current year
- We need the following values for Comfort
- Average temperature/Humidity/CO2 in the last hour (or another time interval more suitable)
Please let me know if I have made myself clear or if you have any doubts.
If you consider it useful, we can arrange a meeting to discuss and clarify some of these points.

Email by Aliki 2023-12-05 about lack of data

Hello all,
Following a thorough investigation into the data posting within the aggregation module, I'm pleased to inform you that we've successfully identified and resolved the root cause behind some entities lacking data or experiencing gaps after the end of October. The issue, traced to a low-level programming aspect which I'm describing below in case you're curious, has been addressed on our end.
I would appreciate it if you could take a moment to test the aggregated data on your side to ensure everything is now functioning as expected. If not, please inform me at your earliest convenience.
The Issue causing the missing data and data gaps:
It appears that an issue arose concerning the data type when submitting information back to the QuantumLeap component. Please refer to the provided examples for clarification:
{"data": [{"id": "AOC-ForumComplex_FirstFloor-NA-NA-Elevators_OdinS-NA-MD0101E20010700105-TH_Humidity_Humidity",
"type": "DeviceMeasurement",
"measurement_type": {"type": "Text", "value": "Humidity"},
"min_value": {"type": "Number", "value": "39.48"},
"max_value": {"type": "Number", "value": "41.15"},
"average": {"type": "Number", "value": "39.874285714285726"},
"total_cost_eur": {"type": "Number", "value": "0.0"},
"kwh_per_square_meter": {"type": "Number", "value": "nan"},
"timestamp_start": {"type": "DateTime", "value": "2023-12-05T12:00:00"},
"timestamp_end": {"type": "DateTime", "value": "2023-12-05T12:59:59"}}]}'
{"data": [{"id": "AOC-ForumComplex_FirstFloor-NA-NA-Elevators_OdinS-NA-MD0101E20010700105-TH_Humidity_Humidity",
"type": "DeviceMeasurement",
"measurement_type": {"type": "Text", "value": "Humidity"},
"min_value": {"type": "Number", "value": "39.48"},
"max_value": {"type": "Number", "value": "41.15"},
"average": {"type": "Number", "value": "39.874285714285726"},
"total_cost_eur": {"type": "Number", "value": "0.0"},
"kwh_per_square_meter": {"type": "Number", "value": "None"},
"timestamp_start": {"type": "DateTime", "value": "2023-12-05T12:00:00"},
"timestamp_end": {"type": "DateTime", "value": "2023-12-05T12:59:59"}}]}'
The only difference between these examples lies in the value of "kwh_per_square_meter", where the first instance contains "nan" and the second instance contains "None". Upon attempting to post this data back to the QuantumLeap component, it was observed that when any attribute had a "nan" value, the data did not appear in the broker. Despite receiving a valid response confirming successful posting, the data was not reflected in the broker.
According to QuantumLeap documentation (https://quantumleap.readthedocs.io/en/latest/user/using/#data-casting), if an entity attribute was in origin received as a Number, following insert changing the same attribute to Text would fail. To mitigate this failures, QuantumLeap attempts data casting, if casting is not possible, values are replaced with None, to ensure that the insert in the database is not failing totally for the received entity.
As a result, the internal QuantumLeap process and our solution unfolds as follows:
1. Handling "nan" Values: When QuantumLeap detects "nan," it attempts to internally cast this value to a numeric type, specifically a "Number". If the internal casting operation fails, QuantumLeap takes a fallback approach and replaces the problematic "nan" value with "None."
2. Possible Internal Processing Issues: Even though a valid response is received, there remains uncertainty about the success of the internal data saving process within QuantumLeap. Due to limitations on our end, it's challenging to verify whether QuantumLeap encounters internal issues preventing the successful storage of data containing "nan."
3. Resolution with "None" Value: Interestingly, changing the value from "nan" to "None" appears to circumvent potential issues during the data insertion process. The use of "None" as a replacement value seems to ensure a smoother operation, possibly overcoming internal challenges associated with handling "nan."
This issue remained unnoticed in numerous entities primarily because a substantial portion of them lacked "nan" values. However, entities housing comfortability data, encompassing temperature, humidity, and CO2, consistently exhibited this problem as their "kwh_per_square_meter" and "total_cost_eur" values were consistently set to "nan".

Email Manuel FC.ID 2023-12-05

Hi Aliki,
Now things seem faster, but we are still not getting any information about comfort.
You can try and see for yourself what is being provided by the API to the application.
By default when we enter (after selecting a building) the application shows the values for "Today". Then we can change the time interval by clicking on the icon on the top right.

Retrieve the queries that WP5 Manuel Fonseca FC.ID is using to query data from the platform. Alternatively, use the credentials provided by FC.ID in order to try the requests interacting with the website to tray to accelerate working in this

We have been monitoring the logs of the QuantumLeap service and also the logs of the 2 PEP-Proxy services of the Smart2B platform, while we were trying to interact with the website "https://project-smart2b.lasige.di.fc.ul.pt/", but no matter what we did, the website always ended up hanging (it seems to be loading information, but it never finishes loading), and none of these interactions with the website have served to register any request to our QuantumLeap service. We have tried to launch some GET requests to QuantumLeap from Postman and these have generated logs, so we deduce that the website hangs before even making any request to QuantumLeap. However, if partners insist that we analyze the response times of our QuantumLeap service, we would need them to tell us what requests they send to our service from the web in order to reproduce it.