2c. Observers' Feedback [WATERVERSE Project Pilots]
  • Functional effectiveness: No major difficulties or gaps were observed.
    Accessibility and Usability: No difficulties in understanding the functionalities of the WDME or the content of the results.

    The results of the Data Collection and Harmonisation tools as well as of the Data Validation tool were easily comprehended and displayed through the WDME tools.

    No visualisation tool was demonstrated.
    Usefulness and acceptance: Positive:
    1) Willingness from SWW's participants to be trained and evaluate the WDME tools. Also, technical partners provide support and materials for training to SWW's personnel.

    2) External observers who participated in this pilot demonstration exhibited their interest in the WDME and expressed the opinion that it is a useful framework for data management. Also, a fruitful discussion on how the WDME can be interoperable with other legacy tools and prediction models was carried out.

    3) The demonstrated WDME tools seem to be relevant to the operations of Water Utility in terms of Data Management and facilitate them in daily operations.
    Learning effectiveness: Strong collaboration among technical partners and pilot operators (users) during the training and rehearsal days.
    Engagement and Enjoyment: No
    Innovation: WDME supports the whole lifecycle of the Water Data Management process providing innovative tools for data harmonisation, validation, processing and analysis of data from heterogeneous sources.
    Cultural appropriateness: No. However, informative pop-ups should be added to the pages of the tools to guide the users.
    Unsolicited Participant Comments: No comments
    Unsolicited observer's comments: No comments
  • Functional effectiveness: Some tools in the DPPE are not currently able to scale to a level which could deal with real-world use cases in terms of volumes and velocities of incoming data.

    It would be good for Iteration 2 if consideration could be given as to how normal users could be enabled to write their own data connectors.
    Accessibility and Usability: Documentation for usage of tools is excellent (e.g. pdfs on sharepoint site). However if would be useful to also have more technical information available so it is clear what the tools are actually doing, and how they work "under the hood" so they are not so much black boxes etc. Perhaps just open sourcing the underlying tool code (with comments) would enable this.

    Many tools when they fail do not return useful error messages to enable debugging - for example "500 internal server error" etc.

    Some of the tools in DPPE do not have associated docs when clicking on the (?) icon, so it is not clear what they do, what they accept as input etc. This makes it hard to consider ways in which DPPE elements could be combined to achieve a data processing goal.

    Usefulness and acceptance: All active participants in the pilot were engaged and very helpful.

    Some tools do not currently scale well to volumes of data we would expect in a real world usage of the tools. This could be problematic if the tools were used at whole of utility network scale.

    It would be good to understand in greater detail what all the other operators in DPPE are capable of, and also ways that a more technical user could create their own operator. (This comment is tied in with earlier comments around documentation etc.)

    Learning effectiveness: I feel a lot more confident in my understanding and use of the tools following the training. In particular Matteo and Sid were able to talk me through the use of the tools and advise me where I was going wrong.
    Engagement and Enjoyment: OK, pilot proceeded smoothly and was concluded ahead of schedule.
    Innovation:
    Cultural appropriateness:
    Unsolicited Participant Comments:
    Unsolicited observer's comments: Overall I think the WDME is progressing well and in the right direction however further development is needed to make it a more scaleable, well documented and adaptable tool.
  • Functional effectiveness: The tool operated as expected.
    Some of the functionalities are more straightforward for the PWN users, while others that need configuration, took longer.
    It appears that challenges were minor.
    It was important that discussion was done on additional functionality or improvement of functionality.
    Accessibility and Usability: The participants were already trained and they new what to do and how.
    However, we need to see how they can generalise the skills and apply things different to the pilot scenario.
    Usefulness and acceptance: People seem enthousiastic and it is obvious that they see the benefit and the potential of WDME.
    Learning effectiveness: Participants were working with shared screen and very interactively. There was obvious support by peers and items were solved collaboratively.
    Participants shared their view on how the tool could be used to address other of their challenges.
    Engagement and Enjoyment: Everybody were focused and to the task. They were all happy to discuss their concerns and their achievements.
    People are also very willing to give feedback to the project.
    Innovation: The system operators are already expecting the value of the WDME and they appreciate it.
    Cultural appropriateness: No issues here, especially because the Netherlands team is very good with English language.
    Unsolicited Participant Comments: Not applicable.
    Unsolicited observer's comments: I am very happy to have participated in the pilot and now I will use this knowledge for the Cyprus pilot where we are technical responsible partner.
  • Functional effectiveness:
    Accessibility and Usability: It seems not so hard
    Usefulness and acceptance:
    Learning effectiveness: participants working together to complete a task
    Engagement and Enjoyment: engaged in the process
    Innovation:
    Cultural appropriateness:
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness:
    Accessibility and Usability: I believe it is pretty easy for the end user to understand the result of a tool or what the visualizations mean.
    The pilot methodology was appropriate and easy to follow.
    Usefulness and acceptance: The pilot sessions were a positive experience, the participants were motivated due the real life uses WDME provides and the overall experience was smooth.
    Learning effectiveness:
    Engagement and Enjoyment:
    Innovation:
    Cultural appropriateness:
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness: At this stage, to my knowledge, there is no agreement on the future use of the WDME (beyond the lifetime of WATERVERSE)
    Accessibility and Usability: The output and way of presenting are quite clear and visual which helps understanding the outcome. It may be worth quantifying the uncertainty of the predictions (based on the underlying data and uncertainties in assumptions made). This may help the end user to make reliable (and timely) decisions on eg temporarily stopping intake of water from the lake.
    Usefulness and acceptance: I see as positive that the original data from external sources is not re-worked or translated in new databases to be used, which secures the system against introducing errors.
    Learning effectiveness: no specific comment
    Engagement and Enjoyment: engagement is fine.
    Innovation: I think it is innovative as it builds on actual questions from end users and uses up to date methologies and insights.
    Cultural appropriateness: I do not think so. The focus is on reliable drinking water which should be relevant for all.
    Unsolicited Participant Comments: no
    Unsolicited observer's comments: no
  • Functional effectiveness: I do not have a observed a gap.
    Accessibility and Usability: I think i could answer this question after i realy use th tool. Know i am more a oberver in the role as productowner digital twin
    Usefulness and acceptance: This is very usefull. There are a lot of PWN collegues and other stakholders interessted . We alreay need it for the complex issues we are deaing with.

    We need to take effort in teaching PWN collegues in the new way of water data management within in outside PWN.
    Learning effectiveness: The pilot with this tool set up a new network within and outside PWN. All the participants are learning a lot and sharing knowledge..
    Engagement and Enjoyment: All the paricipants offline are very interested and willing to be a participant in the next fase of the project
    Innovation: For me the multistakeholder forum is the most innovative part.
    Cultural appropriateness: No
    Unsolicited Participant Comments: No
    Unsolicited observer's comments: No
  • Functional effectiveness: This was my first introduction to the project, in light of my background, I am not able to judge this at the moment.
    Accessibility and Usability: The pilot methodology appears logical.
    Usefulness and acceptance: There were a lot of comments regarding security concerns and authorisation.
    Learning effectiveness:
    Engagement and Enjoyment: The participants appear very motivated and engaged.
    Innovation:
    Cultural appropriateness: I didn't notice any inappropriate metaphors.

    It looks clear and understandable, I do not understand the underlying principles, this is not my field of study.
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness: There is a lack of checks when sending information (i.e. dashboards) directly to the end user. It would be nice to have it send an email to QA or approve before sending it to external stakeholders.
    Accessibility and Usability: Accessibility seems to be easy to use. However, there was a issue where the synthetic data generator was not turned off during the training so it was running for a few days. This would use a lot of processing power. It would be good to have a notification that it is still running.
    Usefulness and acceptance: Concerns about cyber security and concerts about authorization of users.
    Motivation to use this with other water utilities.
    Learning effectiveness: Evidence that behavioral change occurred was evident in the questions asked and looking to increase the scope in the future.
    Engagement and Enjoyment: Participants in the room at PWN were engaged through the presentation.
    Innovation:
    Cultural appropriateness:
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness: The future estimates are based on averages of the past.
    The extreme peaks of the past look not to be incorporated in the prognosis model.
    A way to deal with this could be to create a new metric: (for instance:) the steepness and length of recorded past peaks; this could determine how prepared we should be for deviations.
    Accessibility and Usability: The usability looks really great. Easy to use and access. Access with a click on a link, enticing dashboards with good visuals (statististics and topographic maps).
    Usefulness and acceptance: I see a lot of positive energy from the motivated participants.
    Learning effectiveness: I saw good cooperation between the different participating organisations with a good result. People were helping each other with presenting and answering questions.
    Engagement and Enjoyment: I was happy to be a part of this.
    Innovation: Quite innovative
    Cultural appropriateness: No such thing; there was an appropriate and friendly culture.
    Unsolicited Participant Comments: No
    Unsolicited observer's comments: Still to discover:
    authorization
    authentication
    definitions of private/public
  • Functional effectiveness: This depends on the kind of users. I think you need both a technical (data) background as well as a functional background.

    How do you know what kind of data, datasets, dashboards and or pipelines are activated on the platform? for example if i want to make a graph about chlorade values, how do ik know if this is already made? if i want to use the weather data of germany, how do i know if this dataset is present? How do you know what you don't know?

    beside that, how do is the data quality guarenteed, what kind of metrics and units are used?
    Accessibility and Usability: This again depends on your background. I have a technical data background but even for me it is challenging to follow every step. Especially making a pipeline of building a report or dashboard

    For what i saw i am wondering what the customizing possibilities are. especially for the dashboards. Looking at the dashboard i missed the context, what are the units? when was the data last refreshed, explanation on the x and y axes. What am i looking at and what can you custimize to make things clearer?

    Usefulness and acceptance: The motivation of the participants is very good. They know what they are talking about and they are able to share this in an understandable way.
    Learning effectiveness: I think that building this platform requires a great deal of cooperation between the participants.. I see that the participants have different backgounds, skills and cultural differences and this helps in creating this international multiple used platform. I know for a fact that the participants help each other.
    Engagement and Enjoyment: i haven't noticed any distraction by the participants. They seemed happy with the result and are enthusiastic in sharing this.
    Innovation: no comments. The underlying technoly is not that innovative but using and combining this technology in an international platform is.
    Cultural appropriateness: The participants understand the tools very well. However, for the observers it isn't always clear what we are seeing. especially for the dashboards. Looking at the dashboard i missed the context, what are the units? when was the data last refreshed, explanation on the x and y axes. What am i looking at and what can you custimize to make things clearer?
    Unsolicited Participant Comments: not that i am aware of
    Unsolicited observer's comments: There were some questions about the cyber security. This is obviously a very important aspect.
  • Functional effectiveness: The functionalities are clear and well designed.
    One possible gap that could improve is to have a clear view about who has access to the data and the ability to revoke this at any time and for whatever reason.
    Accessibility and Usability: No difficulties in understanding. The required time for understanding is reasonable (under 1 hour)
    The methodology also seems reasonable, logical and makes sense.
    Usefulness and acceptance: The participants well well motivated and informed.
    The tools and functionalities are well connected to real challenges and there are plans to integrate them more with the real needs and operational environment of the utility.
    The main barrier would be the slow response in procedures (i.e. the conservative attitude) of utilities for the adoption of new tools and technologies.
    Learning effectiveness: The development of the tool progresses very well. There are clear plans about the next steps. The participants work together to achieve results and there is clear support by the more technical savvy people to the less experienced ones.
    Engagement and Enjoyment: No, I have not noticed anything like this.
    Innovation: WDME opens and leads a pathway for the development of thematic European Data Spaces for the Water sector, which is the future for data management, data sharing and data exchange. Additionally it is compatible to the FIWARE platform, which improves its chances for impact
    Cultural appropriateness: No, all is fine.
    Unsolicited Participant Comments: No
    Unsolicited observer's comments: I commented on the improvement of access to the data, which I have already included in my first comment here.
  • Functional effectiveness: Anonymization tool need to be more localized, consider the use of different anonymization techniques also mentioned in https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2014/wp216_en.pdf

    Repetition in filling forms should be minimized to avoid mistakes.

    There should be a way to handle contracts, and create keys to companies that can be revoked in time or on demand.
    Accessibility and Usability: The flow was clear, but at some point it required changing multiple screens (e.g., when getting the API links). Measure the steps in the video and see if they could be reduced..
    Usefulness and acceptance: Participants were motivated, this solves a real problem for data management in water utilities.
    Learning effectiveness: It seems to be knowledge intensive, requires expertise in software/IT as well as operations technologies (OT)
    Engagement and Enjoyment: Positive attitute
    Innovation: This is an innovative tool which will save 100s of workhours, and probably 10,000's of euros when implementing new projects.
    Cultural appropriateness: The "IT" Culture vs the "OT" culture seems to be the biggest issues.
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness: I don't see any description of the functionality. How the uses knew which block to use?
    Accessibility and Usability: The usability can be improved. It seems difficult to me to understand the blocks to be used and added in the flow.
    The pilot methodology seems appropriate.
    Usefulness and acceptance: The used blocks seem cover the required functionality for the pilot.
    Learning effectiveness: The end-user learnt very quickly how to use the WDME and the tools.
    Engagement and Enjoyment: The end-user has been engaged all the time.
    Innovation:
    Cultural appropriateness: No
    Unsolicited Participant Comments: N/A
    Unsolicited observer's comments: N/A
  • Functional effectiveness: Some terminology, such as outliers, time series, or other technical terms could be difficult to understand for non-expert users and stakeholders willing to use these tools. Maybe a little section devoted to explain these terms in simple words could enrich the product.
    Accessibility and Usability:
    Usefulness and acceptance:
    Learning effectiveness: I liked that the bug related to the format of some data (string, not integer) was solved during the demo, putting in value the support of the developers during this session
    Engagement and Enjoyment: I felt that these sessions were not meant for RO developers, as we are not planning to gather data in the future
    Innovation: We like how user friendly the mesh-up editor looks, making it highly usable and appealing
    Cultural appropriateness:
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness: User interfaces need refinement to make it more attractive and user friendly.
    Accessibility and Usability: Challenging. It requires previous trainings to be completed.
    Usefulness and acceptance: Extremely useful and once you get the grasp of it you can understand that there is huge potential with the considered tools.
    Learning effectiveness: Coordination, cooperation and achieving unique goals were all there.
    Engagement and Enjoyment: Quite engaged in the processes. However, a hands on experience would be much more interesting. Maybe, participants can also have a document with instructions as a generic workshop which they can try to follow as well.
    Innovation:
    Cultural appropriateness:
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness: The resolution of the mao could be a bit higher
    Accessibility and Usability: For was easy to understand and the pilot methodology appears appropriate
    Usefulness and acceptance: The participants are motivated and the purpose is addressing a real life challenge in the village.
    Learning effectiveness: I can't comment yet on this.
    Engagement and Enjoyment: can't comment
    Innovation: Comparing to the situation where we are in germany the nature of WDME is very innovative.
    Cultural appropriateness: all fine
    Unsolicited Participant Comments: no, all fine
    Unsolicited observer's comments: no, all fine
  • Functional effectiveness: The retrieval of some datasets was challenging during the trainings.
    Then the operators managed to master the process.
    Accessibility and Usability: Some of the configuration forms of tools require some additional effort to be easily underatood.
    Usefulness and acceptance: There seems to be big relevance of the executed use cases to the real challwnges they face in Etteln. This was explained clearly in the first session of the pilot.
    Learning effectiveness: Excellent collaboration between participants.
    Engagement and Enjoyment: Everybody was concentrated and the process was enjoyable. It helped that people were also physically attending and they managed to have stronger connection to the challenges.
    Innovation: No challenging about the innovation aspects.
    Cultural appropriateness: Looks appropriate.
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness: Etteln Data connection was not succesfull
    Accessibility and Usability: It is hard to understand how to proceed at a glance . There are many screens to give a name.
    Usefulness and acceptance:
    Learning effectiveness:
    Engagement and Enjoyment:
    Innovation: There should be some changes on the UI to facilitate and understand connections
    Cultural appropriateness:
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness: Standardization in terms of naming
    Accessibility and Usability: It is fine to understand. Sometimes it takes a bit too long, making it difficult to keep your attention
    Usefulness and acceptance: The discussions about open source and licenses, about why through firmware or through WDME, are very interesting. The additions from the developers of the tools about what is and isn’t possible are also a good contribution.
    Learning effectiveness: Throughout the process, the operator was well assisted the entire time.
    Engagement and Enjoyment: It takes a bit too long. It is now also repetition for me, which makes you notice that you get distracted very quickly.
    Innovation: For me, it is very innovative. I see many opportunities to apply the currently presented use case to our challenges as well
    Cultural appropriateness: Not applicable
    Unsolicited Participant Comments: Not applicable
    Unsolicited observer's comments: Too long a time between training and rehearsal
  • Functional effectiveness: non so far
    Accessibility and Usability: I am not a technician and I find it understandable and easy to use
    Usefulness and acceptance: I can't answer this question. Everything went smoothly in todays call and the calls I joined
    Learning effectiveness: I am not able to answer this question.
    Engagement and Enjoyment: Everyone seems well prepared and focussed
    Innovation: No
    Cultural appropriateness: I suppose no. I could follow the conversations even with little technical knowledge
    Unsolicited Participant Comments: none
    Unsolicited observer's comments:
  • Functional effectiveness: Allgemeine Fragestellung zur KI-basierten Hochwasservorhersage in Etteln:
    Die Sensoren wurden vor Kurzem installiert. Werden nicht lange Messzeitreihen benötigt, um die KI trainieren zu können? Ab wann kann man mit belsatbaren Ergebnissen rechnen?
    Accessibility and Usability: Auch wenn ich keine Datenspezialistin bin, konnte ich dem Vorgehen gut folgen.
    Die Methodik scheint mir angemessen.
    Usefulness and acceptance: Alle Teilnehmer*innen waren motiviert.
    Learning effectiveness: Alle Teilnehmer*innen waren sehr hilfsbereit.
    Engagement and Enjoyment: Alle Teilnehmer*innen schienen sehr konzentriert.
    Innovation: Hochwasservorhersage mittels KI
    Cultural appropriateness: Nein, das ist mir nicht aufgefallen.
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness:
    Accessibility and Usability:
    Usefulness and acceptance:
    Learning effectiveness:
    Engagement and Enjoyment:
    Innovation:
    Cultural appropriateness:
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness: Cuesta integrar varias bases de datos. Lentitud herramienta.
    Accessibility and Usability: Problemas en general, carezco de información suficiente.
    Usefulness and acceptance: Creo sería útil, pero carezco de información suficiente.
    Learning effectiveness: carezco de información suficiente.
    Engagement and Enjoyment: 13 a 13:30 h explicación somera de la aplicación. No se distrae.
    Innovation: carezco de información suficiente.
    Cultural appropriateness: carezco de información suficiente.
    Unsolicited Participant Comments: No
    Unsolicited observer's comments: carezco de información suficiente.
  • Functional effectiveness: Integrar dos fuentes de datos diferentes.
    Accessibility and Usability: Me falta conocimiento de la integración de datos pero parece intuitiva
    Usefulness and acceptance: Es una plataforma muy interesante para recopilación, aprobación de datos
    Learning effectiveness: No lo conozco
    Engagement and Enjoyment: No lo conozco porque no he pasado mucho tiempo con la herramienta.
    Innovation: Me parece algo muy adecuado tener una herramienta que recoja todos los datos que se necesiten para la gestión.
    Cultural appropriateness: No lo sé
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness: Cuesta integrar dos fuentes de datos diferentes. Lentitud en algunos aspectos de la herramienta.
    Accessibility and Usability: Es una herramienta intuitiva, de fácil manejo, aunque me falta trabajar en ella para evaluar completamente esta respuesta.
    Usefulness and acceptance: Es una plataforma de gran utilidad. Es necesaria la gestión de datos para que la empresa pueda ser competitiva y transparente.
    Learning effectiveness: No aplica.
    Engagement and Enjoyment: La herramienta no da tendencia a distracciones
    Innovation: Análisis de datos ligados a la gestión del agua e inteligencia artificial
    Cultural appropriateness: No conozco
    Unsolicited Participant Comments:
    Unsolicited observer's comments:
  • Functional effectiveness: Dificultad en integrar varias fuentes de datos
    Accessibility and Usability: Es Intuitiva, de fácil adaptación
    Usefulness and acceptance: Gran utilidad para la gestión de datos
    Learning effectiveness: No aplicada
    Engagement and Enjoyment: No aplicada
    Innovation: Puede ser muy interesante
    Cultural appropriateness: Gráficamente Interesante
    Unsolicited Participant Comments: No
    Unsolicited observer's comments: La forma de recoger los datos iniciales me parece correcta.
  • Functional effectiveness: I only attend the first pilot day organize by Keypro. As a integrators the project provide vary methods of connecting and getting data. All of the interface are follow market standard and easy to use.
    Accessibility and Usability: As my personal understanding your tools sismilar to some ETL tools on market. The way of using is similar also, there are some unnecessary clicks but I think overall not too difficult to use.
    Usefulness and acceptance: I think this question is more for water company user.
    Learning effectiveness:
    Engagement and Enjoyment:
    Innovation:
    Cultural appropriateness: No, I think all good.
    Unsolicited Participant Comments:
    Unsolicited observer's comments:

Funding Support Agencies

EU funding support flag
EU structural funds flag
flag of Republic of Cyprus
CY structural funds
CY Research and Innovation Foundation