Assaf, Mohammad (2022) Automated Planning of Data Processing Pipelines. Bachelor's Thesis, Computing Science.
|
Text
Thesis__Automation_and_verification_of_data_pipelines_planning (8) (3).pdf Download (997kB) | Preview |
|
![[img]](https://fse.studenttheses.ub.rug.nl/style/images/fileicons/text.png) |
Text
Toestemming.pdf Restricted to Registered users only Download (124kB) |
Abstract
Large-scale Big Data processing has become imperative as more data either structured or unstructured is being collected especially with the rise of IoT technologies and analytics. This brings demands for processing large chunks of data which brings many challenges such as real-time feedback. Distributed system computing is commonly applied to solve such challenges and often uses data processing pipelines to manipulate the data. Data processing pipelines are sequence of operations and transformations applied to data to achieve a desired task. Planning data processing pipelines is crucial because if not planned properly, it can increase risks and errors that can have consequences such as downtime. However, such planning is a non-trivial labour intensive task that requires consideration of many factors and constraints such as latency and computational resources. In this thesis, I analyse different aspects of data processing and AI planning to construct a theoretical framework that defines different orientations of planning as a scoping strategy. I also leverage the concept of Object-relational mapping (ORM) to easily generate Planning Domain Definition Language (PDDL) planning problems. I use the framework to develop an AI-powered data processing pipeline planner that utilises the Fast Downward planning system and PDDL ORM. The planner can be used to construct valid pipelines that achieve desired data tasks. Unlike other data pipeline planners, this planner can be easily extende
| Item Type: | Thesis (Bachelor's Thesis) |
|---|---|
| Supervisor name: | Degeler, V. and Hadadian Nejad Yousefi, M. |
| Degree programme: | Computing Science |
| Thesis type: | Bachelor's Thesis |
| Language: | English |
| Date Deposited: | 21 Jul 2022 06:03 |
| Last Modified: | 21 Jul 2022 06:03 |
| URI: | https://fse.studenttheses.ub.rug.nl/id/eprint/28077 |
Actions (login required)
 |
View Item |