Periodic Reporting for period 2 - RECORD-IT (Reservoir Computing with Real-time Data for future IT)
Période du rapport: 2016-09-01 au 2018-08-31
In principle, any sufficiently complex dynamical system can be used for universal computation, as a reservoir computer, if it can be equipped by an adjustable readout layer. The overall aim of the project is to demonstrate a novel class of sensing approaches, where the reservoir both collects and processes information at the same time, being an intelligent sensing substrate. We suggested and developed a rigorous mathematical formalization of this idea, to be referred to as the State Weaving Environment Echo Tracker (SWEET) sensing framework/algorithm.
The sensing element and the environment are treated conceptually as one dynamical system, a super-reservoir with states what are weaved together (entangled). The reservoir functions as the state weaver so that the state of the super-reservoir at a particular time instance embodies everything that both the weaver and the environment have experienced in the past (the echo state property). We develop ways of analyzing the reservoir/weaver states to infer indirectly about the environment (indirect sensing through optimized drive).
We provided theoretical and experimental demonstrators of the SWEET/RECORD-IT principle to detect and classify complex behavioral changes in ion concentrations. The two major objectives are: (O1) Build a reservoir computation device prototype and (O2) Explore the capabilities of the device.
We found a very generic way of extracting the information stored in the weaver, the SWEET algorithm. The input signal is used to achieve supervised learning through phase space separation. The delayed feedback mechanism is used to enrich the structure of the waver states and to keep the input and the environmental signals synchronized.
We develop an entirely novel concept for sensing which is beautifully generic, extremely open ended, and scalable in terms of the number of sensors, or sensor types.
The sensing element and the environment are treated conceptually as one dynamical system, a super-reservoir with states what are weaved together (entangled). The reservoir functions as the state weaver so that the state of the super-reservoir at a particular time instance embodies everything that both the weaver and the environment have experienced in the past (the echo state property). We develop ways of analyzing the reservoir/weaver states to infer indirectly about the environment (indirect sensing through optimized drive).
We provided theoretical and experimental demonstrators of the SWEET/RECORD-IT principle to detect and classify complex behavioral changes in ion concentrations. The two major objectives are: (O1) Build a reservoir computation device prototype and (O2) Explore the capabilities of the device.
We found a very generic way of extracting the information stored in the weaver, the SWEET algorithm. The input signal is used to achieve supervised learning through phase space separation. The delayed feedback mechanism is used to enrich the structure of the waver states and to keep the input and the environmental signals synchronized.
We develop an entirely novel concept for sensing which is beautifully generic, extremely open ended, and scalable in terms of the number of sensors, or sensor types.
In the project we have carefully balanced theoretical and experimental efforts and always ensured a maximum synergy between the two. On the theory side we have developed an advanced simulator that can be used to perform virtual experiment on a computer. We have developed plethora of models for the devices used in the project and implemented these on a computer. The theoretical efforts include a range of supporting activities: the first principle calculations, theoretical physics models of various components and behavior (ion binding kinetics and diffusion).
In the experiments, we have developed a range of devices that are sensitive to the changes in the ionic concentration. We have also developed an advanced ion delivery system to simulate various ion concentration profiles experimentally. Several of the devices have been integrated. We have achieved a range of homo-integration options in the forms of arrays of similar elements. We have also developed heterointegration methods that can be used to couple several sensors.
The main results include:
(1) Theoretical validation the SWEET/RECORD-IT sensing concept:
(1.a) We have validated the concept theoretically through numerical simulations. The idea of indirect sensing through applied drive to achieve phase space separation works very well. For the classification problem we have investigated, optimizing the drive is equivalent to supervised learning.
(1.b.) The heterogeneity of the sensor components is of paramount importance. The heterogeneity of the components and the delayed feedback ensure the separation property, which guarantees a good sensing reservoir.
(1.c) We have formulated an important bar-code principle, as an example how the indirect sensing can be engineered in practice.
(2) We carried out a series of experimental demonstrations:
(2.a) We have developed an advanced and programmable open microfluidic ion delivery system that can simulate arbitrary time dependent concentration profiles (homogenous in space but changing in time).
(2.b) Ours OECT device is an innovative approach that relies both on the intrinsic physics (ionic dynamics) of OECTs and on the neuromorphic calculation concepts of the “reservoir computing” type (i.e. a spatio-temporal data processing in a network with complex dynamics and strong non-linearity) to demonstrate learning and event classification (square versus triangular waves).
(2.c) We have demonstrated pH sensing using SiNW device with the fading memory behavior. The transient response of the system exhibits a strong dependence on the ion concentration (KCl electrolyte, Cu ions)
(2.d) We have demonstrated the use of the delayed feedback mechanism that generates hyperloops in the response curves at the edge of chaos, and heterointegration options by combining photoelectrochemical sensors and cation-sensitive electrochemical transistors. Our approach can be used to increase the sensitivity and improve the detection limit of conventional electrochemical (or photoelectrochemical sensors) and offers a universal platform for signal processing within the sensors themselves.
(2.e) We have develop chemically extremely complex biomimetic sensing device that uses the oxytocine hormone to function, being very sensitive to the spatial distribution of the ions and to the diffusion of ions to and away from the electrode. We have tested the device for advanced medical applications in the context of brain disorders.
In the experiments, we have developed a range of devices that are sensitive to the changes in the ionic concentration. We have also developed an advanced ion delivery system to simulate various ion concentration profiles experimentally. Several of the devices have been integrated. We have achieved a range of homo-integration options in the forms of arrays of similar elements. We have also developed heterointegration methods that can be used to couple several sensors.
The main results include:
(1) Theoretical validation the SWEET/RECORD-IT sensing concept:
(1.a) We have validated the concept theoretically through numerical simulations. The idea of indirect sensing through applied drive to achieve phase space separation works very well. For the classification problem we have investigated, optimizing the drive is equivalent to supervised learning.
(1.b.) The heterogeneity of the sensor components is of paramount importance. The heterogeneity of the components and the delayed feedback ensure the separation property, which guarantees a good sensing reservoir.
(1.c) We have formulated an important bar-code principle, as an example how the indirect sensing can be engineered in practice.
(2) We carried out a series of experimental demonstrations:
(2.a) We have developed an advanced and programmable open microfluidic ion delivery system that can simulate arbitrary time dependent concentration profiles (homogenous in space but changing in time).
(2.b) Ours OECT device is an innovative approach that relies both on the intrinsic physics (ionic dynamics) of OECTs and on the neuromorphic calculation concepts of the “reservoir computing” type (i.e. a spatio-temporal data processing in a network with complex dynamics and strong non-linearity) to demonstrate learning and event classification (square versus triangular waves).
(2.c) We have demonstrated pH sensing using SiNW device with the fading memory behavior. The transient response of the system exhibits a strong dependence on the ion concentration (KCl electrolyte, Cu ions)
(2.d) We have demonstrated the use of the delayed feedback mechanism that generates hyperloops in the response curves at the edge of chaos, and heterointegration options by combining photoelectrochemical sensors and cation-sensitive electrochemical transistors. Our approach can be used to increase the sensitivity and improve the detection limit of conventional electrochemical (or photoelectrochemical sensors) and offers a universal platform for signal processing within the sensors themselves.
(2.e) We have develop chemically extremely complex biomimetic sensing device that uses the oxytocine hormone to function, being very sensitive to the spatial distribution of the ions and to the diffusion of ions to and away from the electrode. We have tested the device for advanced medical applications in the context of brain disorders.
In the traditional sensing setup, if used, the reservoir if used passively. In such “reservoir computing and sensing” approaches the reservoir is used to post-process the information that is generated by a separate sensor component. The reservoir can be safely replaced by an artificial neural network without any impact on the functionality of the sensor. However, in our “reservoir computing for sensing” approach, the reservoir is an integral part of the sensor. It cannot be replaced without worsening the performance of the sensor.
In the traditional sensing setup, the sensor-environment interaction has to be carefully engineered and it should be strong (to beat noise). In our setup, the sensor-environments interaction can be weak, not fully known, it does not be carefully engineered, and it can be possibly random, being suitable for embedded biotech IoT sensing applications.
The application possibilities and the socio-economic impact is enormous since sensors are ubiquitous in nearly all technological applications. We have demonstrated the concept within the academic environment. Our dissemination activities should make the ideas accessible to various types of potential lead users.
The novel way of sensing being developed in the project can be used to both design novel sensing applications, but also to re-use the existing ones in unexpected ways. We anticipate further impact on other industries, since the algorithm is universally applicable to several societal challenges where advanced neuromorphic, embedded, and unconventional information processing techniques could be used with a great advantage (energy consumption control, traffic jam prediction, transport logistic planning, security, medical applications, etc.)
In the traditional sensing setup, the sensor-environment interaction has to be carefully engineered and it should be strong (to beat noise). In our setup, the sensor-environments interaction can be weak, not fully known, it does not be carefully engineered, and it can be possibly random, being suitable for embedded biotech IoT sensing applications.
The application possibilities and the socio-economic impact is enormous since sensors are ubiquitous in nearly all technological applications. We have demonstrated the concept within the academic environment. Our dissemination activities should make the ideas accessible to various types of potential lead users.
The novel way of sensing being developed in the project can be used to both design novel sensing applications, but also to re-use the existing ones in unexpected ways. We anticipate further impact on other industries, since the algorithm is universally applicable to several societal challenges where advanced neuromorphic, embedded, and unconventional information processing techniques could be used with a great advantage (energy consumption control, traffic jam prediction, transport logistic planning, security, medical applications, etc.)