python - Generic Regex building to match URL routes based on its constant and variable parts

Question

Suppose we are intending to scrape a domain, foo.com, which virtually contains many external sublinks, such as:

• /bar/1020/biz?foo=qwe
• /bar/3001/biz?foo=rty
• /bar/12033/biz?foo=abc
• /bar/1020/biz?foo=xyz

The idea is to apply a difference checker algorithm which yields a generic regex that matches the above routes based on their constant and variable parts.

In this case, the perfect regex would be:

^/bar/d+/biz?foo=.*?$

Which in other words implies:

/bar/{any number}/biz?foo={any string}

What is the best approach to achieve the following, given that a dataset of 100-ish routes are provided?

question from:https://stackoverflow.com/questions/65842460/generic-regex-building-to-match-url-routes-based-on-its-constant-and-variable-pa

Answer 1

Waitting for answers