Serial Pipeline Tutorial

The Serial Pipeline in Syncraft is designed to execute a series of pipelines in a sequential manner. Each step in the sequence is executed one after the other, with the output of one step feeding into the input of the next. This sequential execution model is perfect for scenarios where data processing needs to occur in a specific order.

Core Components:

Sequence of Pipelines: An ordered list of pipelines to be executed sequentially.
Data Flow: The data flows from one pipeline to the next in the specified order.

Try In Playground


[
    {
        "query": {
            "id": {
                "eq": 2
            }
        },
        "selection": {
            "id": true,
            "name": true
        },
        "root": "Users"
    },
    {
        "query": {
            "UserID": {
                "eq": {
                    "@ref": ["id"]
                }
            },
            "ActionType": {
                "eq": "PURCHASE"
            }
        },
        "selection": {
            "ActionType": true,
            "Timestamp": true
        },
        "root": "SessionRecord"
    }
]

Workflow:

Initial Query: The first pipeline queries the Users root for a user with id equal to 2, selecting the id and name fields.
Action Retrieval: The second pipeline in the sequence takes the id from the first pipeline's output and queries the SessionRecord root for actions by that user where action was "PURCHASE".

By structuring the query logic in a serial manner, developers can create organized, step-by-step data retrieval and transformation workflows. This not only makes the logic easier to follow but also ensures that data is processed in the correct order, making the Serial Pipeline a vital tool for handling multi-step data processing tasks.

Learn More

Looping Pipelines

Continue Your Journey

Building Parallel Pipelines