Functional Pipeline

URL:

Pypi: https://pypi.org/project/functional-pipeline/

Gitlab: https://gitlab.com/mc706/functional-pipeline

Description

A tool to make writing pipelines of functions and composing functions to write and read. I wrote a series on the development on this:

In a nutshell it allows you to replace nosy code like:

1
2
3
4
5
6
starting = get_data_from_remote()
result_1 = process_step_1(starting)
result_2 = process_step_2(result_1)
result_3 = process_steo_3(result_2)
result_4 = process_step_4(result_3)
final = = process_final_result(result_4)

or possibly more cleanly without the intermediate values:

1
2
3
4
5
6
7
8
9
10
11
process_final_result(
    process_step_4(
        process_step_3(
            process_step_2(
                process_step_1(
                    get_data_from_remote
                )
            )
        )
    )
)

into clean and easier to reason about code like:

1
2
3
4
5
6
7
8
9
10
final = pipeline(
    get_data_from_remote(),
    [
        process_step_1,
        process_step_2,
        process_step_3,
        process_step_4,
        process_final_result,
    ]
)

It also has first class support for generators allowing for lazy evaluation and trivial support for parallelism using multiprocessing.Pool.map

Technologies:

  • Python
  • Function Composition

Setup

1
$ pip install functional-pipeline

Documentation

ReadTheDocs