Control and Reinforcement Learning in Stochastic Processes

Control and Reinforcement Learning in Stochastic Processes
Business and Territorial Competitiveness, Innovation and Sustainability
The Chair of Computational Mathematics (CCM) of Deusto Foundation – University of Deusto (Bilbao, Basque Country, Spain), aims to develop an active research, training and outreach agenda in the broad field of Applied Mathematics.
Lead by Prof Enrique Zuazua, and funded, among other agencies, by the Advanced Grant DyCon of the European Research Council (ERC), the Chair is committed with the development of ground-breaking research in the areas of Partial Differential Equations, Control Theory, Numerical Analysis, Data Sciences and Scientific Computing; key tools for technological transfer and for the interaction with other scientific disciplines such as Biology, Social Sciences, Engineering, Earth, Social and Climate Sciences.
The thesis will be co-supervised by Jose María Amigo Garcia, from the Center of Operation Research (CIO in Spanish) of the Universitas Miguel Hernandez (Elche, Alicante, Spain). Dr. Amigo is the Director of the Dynamical systems and applications research group in the Area of Applied Mathematics. His research comprises the following: (1) Dynamical systems in discrete time: complexity in dynamical systems, synchronization, control of internet data congestion. (2) Ergodic theory: permutation entropy, topological entropy, transfer entropy, Kolmogorov-Sinai entropy, applications to chaotic cryptography; (3) Information theory: estimation of Shannon entropy, applications to computational neuroscience. (4) Nonlinear time series analysis: complexity, causality, prediction. (5) Mathematical physics: operator theory, computational fluid mechanics.
The Chair of Computational Mathematics (CCM) develops its activity in the Deusto Foundation – University of Deusto (Bilbao, Basque Country, Spain), oriented to computer, data and mathematical sciences, with excellent infrastructure and embedded in an international network of elite academic and research institutions.
TThe CIO was created in 2005 with the aim of supporting the development and practice of research in Statistics, Applied Mathematics, Optimization and Information Technologies. The CIO is made up of more than forty researchers from the Miguel Hernández University of Elche (UMH), experts in disciplines such as statistics, mathematics and computing, forming a qualified interdisciplinary group capable of addressing different current problems in the academic and business world. Among its priority objectives is, first, both basic and applied research conducted by its researchers; second, the transfer of knowledge and technology to society, always seeking the improvement of quality, efficiency and productivity in both public institutions as in private companies; and, third, to promote postgraduate training (advanced courses, master’s degrees and doctoral programs).
The candidate should have a master diploma in mathematics, physics, engineering or computer sciences and basic skills in scientific programming (R, python or MATLAB programming languages). A basic knowledge of stochastic processes, such as Markov chains or stochastic differential equations, and dynamic optimization (optimal control, dynamic programming, calculation of variations), would be desirable too.
The candidate should be highly motivated to develop research in dynamic programming, employing databases and data analysis techniques.
  • Mathematics (MAT)
The Chair aims to integrate a PhD candidate to carry out mathematical and computational research in the field of recommendation systems by control.
Recommendation systems are responsible for ordering different elements in the digital age. We interact daily, directly or indirectly, with recommendation systems that influence to a large extent which news we read, movies we watch, or music we listen.
A recommendation system is an algorithm capable of ordering a set of elements and selecting the first ones to maximize a function that represents the valuation of users. It can be interpreted as a dynamic control system, where the recommendations play the role of controls. In this way, different strategies can be designed to maximize the user experience, leading to an innovative approach to this challenging topic.

The Chair CMM aims to employ the classical paradigm of Mathematical “modelling + analysis+ simulation + control “to address these issues, in combination with the recently developed “data driven” techniques.

The main challenge consists in modelling these processes through a combination of classical mathematical tools and data exploration.

EXCELLENCE OF THE HOST RESEARCH UNIT

Privat Y., Trélat E., Zuazua E. – Randomised observation, control and stabilization of waves – J Appl Math Mech Z Angew Math Mech. Vol. 96, No. 5 (2016), pp. 538-549, DOI: 10.1002/zamm.201500181,
U. Biccari, D. Ko, E. Zuazua – Dynamics and control for multi-agent networked systems: a finite difference approach – Math. Models Methods Appl. Sci., Vol. 29, No. 4 (2019), pp. 755–790. DOI: 10.1142/S0218202519400050,
D. Ruiz-Balet, E. Zuazua – A parabolic approach to the control of opinion spreading – In: Berezovski A., Soomere T. (eds) Applied Wave Mathematics II. Mathematics of Planet Earth, Vol 6. Springer, Cham (2019), pp. 343-363. DOI: 10.1007/978-3-030-29951-4_15,
Lu Q., Zuazua E. – Averaged controllability for random evolution Partial Differential Equations – J MATH PURE APPL. Vol. 105, No. 3 (2016)., pp. 367–414. DOI: 10.1016/j.matpur.2015.11.004,
Hernández-Santamaría V., Lazar M., Zuazua E. – Greedy optimal control for elliptic problems and its application to turnpike problems
NUMER MATH. Vol. 141, No. 2 (2019), pp.455-493. DOI: 10.1007/s00211-018-1005-z,
• ERC-ADG-2015 ERC Advanced Grant (694126) DyCon-Dynamic Control Dynamic Control and Numerics of Partial Differential Equations. Partners: Fundación Deusto (CCM)-coordination & UAM_Madrid
• MINECO RETOS 2017. (MTM2017-92996-C2-1-R). Control y estabilidad de redes híbridas AC/DC: Ecuaciones Diferenciales y Ecuaciones en Derivadas Parciales para el análisis de estabilidad de redes (COSNET). Partners: Fundación Deusto (CCM)-coordination, and Mondragon University.
• ELKARTEK 2018 (Basque Government) (KK-2018/00083) «New Tools for the Design and Control of Hybrid ac/dc Distribution Networks (Road2DC)». Partners Mondragon University, Fundación Deusto, TECNALIA, Ingeteam & IK4-IKERLAN

INTERDISCIPLINARY COLLABORATION

The Chair of Computational Mathematics (CCM) incorporates a wide spectrum of expertise and knowledge amongst their team members in areas such as Mathematics, Physics, Engineering and Computer Sciences, and seeks for new collaborative challenges with other public and private entities. In this way, the hosting institution offers the possibility to integrate knowledge in relation to health science, environment, energy and industrial applications amongst others.The Chair of Computational Mathematics (CCM) incorporates a wide spectrum of expertise and knowledge amongst their team members in areas such as Mathematics, Physics, Engineering and Computer Sciences, and seeks for new collaborative challenges with other public and private entities. In this way, the hosting institution offers the possibility to integrate knowledge in relation to health science, environment, energy and industrial applications amongst others.
It will be decided together with the candidate according to his/her specific research situation, thesis proposal and the interdisciplinary aspects of the research project.
Interdisciplinarity is a crucial step planned within the research of the Chair of Computational Mathematics (CCM) whose main aim is the development of new mathematical and computational methods to be implemented in real case scenarios, in particular, through the employment of mathematical modelling in industrial problems related to dynamic control scenarios. To date, the interdisciplinary activities have been planned and implemented actively, taking part in other actions in the field of big data analysis, image treatment and machine learning applications. The research road map of the Chair offers the possibility to enhance this approach by integrating new PhD students, in collaboration with other regional and international research centres.
Aligned with the 2030 Agenda, the Horizon Europe missions and goals and the Basque Country Smart Specialisation Strategy, DEUSTO research blends competitiveness, innovation and technology to tackle challenges for communities, companies and public bodies in the region. Ageing and Wellbeing; Cultural and Creative Industries and Cities; Gender; Social Justice and Inclusion and Strengthening Participation are the focus of the 5 Interdisciplinary Research Platforms. Besides, advanced research units contribute to generate knowledge and innovative solutions in the fields of efficient and clean Energy, Industry 4.0 (advanced manufacturing, data analytics…), Health and Territory.
https://www.deusto.es/cs/Satellite/deustoresearch/es/inicio/plataformas-interdisciplinares-de-investigacion

INTERNATIONAL COLLABORATION

One of the core strategies of the Research Group is to implement an effective international research collaboration plan already carried out over the last 4 years in Deusto Foundation. The research framework of the Hosting offer covers the potential to build direct and indirect relationship with H2020 priorities such as climate resilient future, multi-culturalism, as well as digitising and transforming European Industry and services. The strength of Mathematics lies in its potential to be used in the most diverse disciplines and applications. Mathematics is used to measure, model, predict, quantify, calculate, simulate … And that enormous potential has been amplified in the last decades thanks to the symbiosis with its most advantaged descendants, Computer Science and Modelling. The Chair develops a multidisciplinary training and research agenda in the area of systems control, with applications in very diverse sectors. At the present there is clear relation of the projects to H2020 priorities, Sustainable Development Goals (SGD) : Goal 7: Clean energy, Goal 9: Industry, Innovation and Infrastructure and Goal 17: Partnerships for the Goals. The Chair challenges the mathematical perspective of the development of optimal control strategies in large temporal horizons, particularly important for society. Energy and Environment, Epidemiology, neurodegenerative diseases, wealth distribution, multiculturalism and immigration are issues in which Mathematics, can play a decisive role.
It will be decided together with the candidate according to his/her specific research situation, thesis proposal and the interdisciplinary aspects of the research project. The Chair develops an active international agenda, in particular with the collaboration of the Chair of Applied Analysis-Alexander von Humboldt Professorship at FAU-Erlangen (Germany).

INTERSECTORAL COLLABORATION

Relevant external stakeholders to collaborate with in the framework of the proposal/project could be the Friedrich-Alexander-Universität (FAU) in Erlangen-Nürnberg, in Germany and Universidad Autónoma Madrid (UAM), Spain. The research group already foresees a direct collaboration with private companies facing the need to evaluate the effectiveness of dynamic control theories on real case scenarios for data driven analysis and reinforcement learning. Certain industries have already indicated interest on this research line.
Deusto Research works closely with more than 1250 organisations (public administration, business, non-profit organisations, education and training institutions, technology centres, etc.) in areas such as health, urban development, gender, social justice and inclusion, democratic participation, regional competitiveness and energy. Since 2014 DEUSTO has been involved in 89 international research projects and worked with more than 770 partner organisations. From data collection to the implementation and piloting phases, stakeholders have an active role throughout the research project life cycle.
https://www.deusto.es/cs/Satellite/deustoresearch/es/inicio/transferencia-3/colaboracion-con-agentes-externos

IMPACT

The hosting research group has developed over the last few years a clear Dissemination Plan for the scientific publications as well as a Communication Plan for dissemination of the research approach for a wider audience and targeted groups. There is a clear contribution of the expert knowledge on the attraction of talent to mathematical studies and more all for its application and uses in different societal challenges. The participation of the researchers in scientific events as well as talent attraction events al local and/or international level is key for the exploitation of the mathematical result uptakes and potential impact and contribution by the platform tool user community.
The implementation of the reinforcement learning methodology has the potential to develop specific recommender system tools to enhance the uptake of information and services by final users. The contribution of the proposed topic for PhD to the knowledge field beyond the scientific impact is very relevant, as the recommender system could be potentially adapted to different users’ profile and needs covering from general audiences (citizenship) to special need groups. In principle the research line will define a series group of target stakeholders which could uptake exploit the foreseen results.
Deusto Research pursues societal impact that goes beyond academia focusing on the transformation of today’s society, leading to fairer and more diverse societies, where inclusive social development and welfare are enhanced.
https://www.deusto.es/cs/Satellite/deustoresearch/en/home/dissemination-and-transfer

INNOVATION

The Chair of Computational Mathematics has been conceived and run in a holistic manner, integrating from the very early the research efforts in the mathematical aspects and the software development. This has been done, to increase the potential of interaction of the team with researchers of other fields and other players in the broad European space of Research and Innovation. In addition, the Chair has included the research environment as well where the Principal Investigator, senior lecturers, post-doctoral researches and PhD and Master students share the key aspects of their knowledge as well the expertise to the rest of team. In terms of the innovation related to the research proposal, classic recommendation systems can recognize certain preferences in users. However, they fall into redundancy, preventing users to be offered new attractive elements. This is because most of the recommendation systems assume that the users’ preferences are static. The proposed new methodology will allow the evolution of the user learning from their behavior.
DEUSTO research internationalisation strategy pursues innovation through the participation in international research and innovation initiatives (H2020 projects, Innovation Radar)
https://www.deusto.es/cs/Satellite/deustoresearch/en/home/transfer-of-knowledge-0/innovation-radar

INCLUSION

Inclusion is an underlying principle of the 6i Dirs project. The leadership vocation of the University of Deusto goes hand in hand with its aspiration to excellence and quality and a firm commitment to social justice and inclusion. The Interdisciplinary Platform on Social Justice and Inclusion articulates interdisciplinary collaborations and channel research efforts for contributing to social justice and more inclusive and fairer societies.
https://www.deusto.es/cs/Satellite/deustoresearch/es/inicio/plataformas-interdisciplinares-de-investigacion/deusto-social-justice-and-inclusion/the-platform
Deusto recognises gender equality as a key driver for sustainable development and inclusive growth across regions, and believes that equality of academic opportunity for women is key. The Interdisciplinary Research Platform on Gender is a dynamic vehicle that teams up experts from different areas of knowledge with the dual purpose of fostering collaboration and integrating existing expertise to address society’s emerging challenges on gender issues. (https://www.deusto.es/cs/Satellite/deustoresearch/es/inicio/plataformas-interdisciplinares-de-investigacion/deusto-gender/the-university-of-deusto)
Moreover, to seek real inclusion for people with specific support needs, the 6i Dirs project ensures equal rights and opportunities with respect to access to the programme and the acquisition of the skills expected to achieve the PhD and enhance their career development and future employability. All vacancies on the Euraxess EU portal bear the Science4refugee logo to encourage refugees to apply for them. Furthermore, DEUSTO is a partner of the Scholars At Risk Network (SAR), which offers safety to scholars facing grave threats.