The fact that you have large diffs between branches is not intrinsically a problem. Git stores data as a set of snapshots of the repository at each commit, not as a series of diffs between branches. It does, when packing, store individual objects as deltas against other similar objects, but unless you are storing many binary objects, this usually isn't a problem. So you're probably fine.
However, the approach of keeping one main branch and cherry-picking is, as you've found out, a bit of a maintenance nightmare. As you've hinted at, it would be better to have one unified code base with some configurability plus a set of configuration options for each version.
If you go that route, then you can either have just a main branch with a directory or directories full of config files, or you can have a main branch with a separate branch that differs only in its config file. With the latter approach, your best best is to just merge the main branch in instead of cherry-picking, or rebase your subsidiary branches onto the main branch. Rebased branches can be tricky for others to work with, so I recommend the merge approach.
However, if you insist on cherry-picking, then in the scenario you mention, most of your code isn't going to differ, so git cherry-pick
isn't likely to have a problem. The history of a branch beyond the merge base isn't relevant when you're merging or cherry-picking, so having a nasty history shouldn't be a problem.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…