Garcia Dachary, Ignacio (2023) Design of a monitoring system for distributed data processing pipelines - ECiDA platform monitoring. Master's Thesis / Essay, Computing Science.
|
Text
REPORT_MThesis_IgnacioGarciaDachary.pdf Download (2MB) | Preview |
|
![[img]](https://fse.studenttheses.ub.rug.nl/style/images/fileicons/text.png) |
Text
toestemming.pdf Restricted to Registered users only Download (129kB) |
Abstract
Distributed data processing pipelines allow businesses to reduce the time needed to get models in production, fostering accelerated innovation and breakthroughs in many fields. Monitoring is crucial for ensuring the health and optimal performance of these systems. This thesis aims to provide a blueprint for designing a monitoring system for distributed data processing systems, specifically targeting the ECiDA platform, which facilitates the design and deployment of such pipelines. A literature and documentation review was done to define how these systems should be monitored. Then, a software architecture for the monitoring system was defined and, by reviewing and comparing available open-source tools, a possible implementation of the defined architecture was proposed. It has been observed that monitoring is highly dependent on the specific use case, problem nature, data format, and model type, among other factors. However, the proposed architecture can serve as a foundation for developing a monitoring system for systems like ECiDA, with minor implementation details varying based on each specific problem and context.
| Item Type: | Thesis (Master's Thesis / Essay) |
|---|---|
| Supervisor name: | Lazovik, A. and Hadadian Nejad Yousefi, M. |
| Degree programme: | Computing Science |
| Thesis type: | Master's Thesis / Essay |
| Language: | English |
| Date Deposited: | 18 Sep 2023 07:55 |
| Last Modified: | 18 Sep 2023 07:55 |
| URI: | https://fse.studenttheses.ub.rug.nl/id/eprint/31448 |
Actions (login required)
 |
View Item |