How to split duplicated separator in Python

Question

I have a string with the format exp = '(( 200 + (4 * 3.14)) / ( 2 ** 3 ))'

I would like to separate the string into tokens by using re.split() and include the separators as well. However, I am not able to split ** together and eventually being split by * instead.

This is my code: tokens = re.split(r'([+|-|**?|/|(|)])',exp)

My Output (wrong):

['(', '(', '200', '+', '(', '4', '*', '3.14', ')', ')', '/', '(', '2', '*', '*', '3', ')', ')']

I would like to ask is there a way for me to split the separators between * and **? Thank you so much!

Desired Output:

['(', '(', '200', '+', '(', '4', '*', '3.14', ')', ')', '/', '(', '2', '**', '3', ')', ')']

question from:https://stackoverflow.com/questions/65858996/how-to-split-duplicated-separator-in-python

Answer 1

Using the [...] notation only allows you to specify individual characters. To get variable sized alternate patterns you need to use the | operator outside of these brackets. This also means that you need to escape the regular expression operators and that you need to place the longer patterns before the shorter ones (i.e. ** before *)

tokens  = re.split(r'(**|*|+|-|/|(|))',exp)

or even shorter:

tokens = re.split(r'(**|[*+-/()])',exp)