Adding a pipe operator to Python

The other day, I was looking at some Python code that was made up of a sequence of deeply-nested function calls and a thought immediately got pushed up brain’s stack:

Wouldn’t it be cool if Python had an operator akin to Elixir’s pipe operator?

Pipe operator?

For those of you not familiar with Elixir, the pipe operator |> can be used to turn code that looks like this:

Nested function calls.

Into code that looks like this:

Refreshing!

In essence, the pipe operator takes the expression on its left-hand side and moves it into the first argument position of the expression on its right-hand side, assuming the expression on the right is a function call.

Doing it in Python

Although adding an actual pipe operator to the language isn’t possible without changing the Python interpreter, there’s nothing preventing us from re-purposing an existing operator! That’s exactly what I set out to do.

Borrowing from shells, I figured the “bitwise or” (aka “pipe”) operator would be a good fit for this sort of functionality:

The simplest way I could come up with to repurpose the pipe operator was to rewrite functions using a decorator. Python’s built-in ast module makes this particularly easy.

All I had to do was

• get a function’s source code (using inspect.getsource),
• turn it into an abstract syntax tree by passing it toast.parse,
• walk the tree and transform any occurrences of the pipe operator according to the rules I outlined at the end of the last section and,
• finally, compile and return the rewritten function object.

That sounds a lot more complicated than it turns out to be in practice.

The ast module provides a class called NodeTransformer that implements the visitor pattern: its visit method does a depth-first search through an AST and calls any declared methods of the form visit_NODETYPE on itself for each node in the tree. As the name implies, you can use a node transformer to visit and manipulate the nodes of an AST:

Whenever the transformer encounters a binary operator, it recurses so that any transformations that need to be made on its left-hand and right-hand nodes are made first, then it checks if the current operator is “bitwise or”.

If the current node’s operator is “bitwise or” and if the right-hand side is a function call node, then it inserts the left-hand side into the first argument position of the function call and then returns the node on the right-hand side, replacing the binary operator node with the call node in the tree.

The transformer also kicks in when it sees a function definition so that it may remove the enable_threadop decorator. This’ll make sense once you take a look at the decorator itself:

The decorator takes a function as an argument, grabs its source code and removes any indentation (this is important! otherwise, decorating class methods results in a SyntaxError), parses the code and transforms the AST and, finally, it compiles and executes the function definition, returning the resulting function object.

If the transformer hadn’t removed the decorator from the final tree, then we’d have an infinite loop on our hands because enable_threadop would be called over and over again when the function definition is executed (line 13).

With all that in place, the enable_threadop decorator may be used to selectively change the behaviour of the pipe operator:

If that’s not a realistic example, I don’t know what is!Limitations

As you would expect, there are a couple limitations with this approach.

First off, inspect.getsource grabs functions’ source code off of the file system, meaning that the decorator won’t work in the Python interpreter.

Secondly, the transformer requires the right-hand side of the pipe operator to be a function call.

Time to use it in production!

Whoa there, slow down! This is just a neat little experiment and is most definitely not something you should inflict upon your coworkers!

That said, if you want to play around with it, you can find the full code (50 lines with comments!) on GitHub.

Thanks for reading! If you liked the article, give it a clap! You can also find me on my website at https://defn.io, on GitHub and Twitter.

Bogdan Popa (@bogdanp) | Twitter

Adding a pipe operator to Python was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.