Nah. The navigation keys on the home row are a throwback to when terminals were less likely to handle control keys correctly. If that's not a problem you have, then don't worry about it.

You'll probably benefit from getting the commands to move by paragraphs, functions, pairs of () {} <>,

One of my coworkers uses only C-[pnbf] for cursor movement in Emacs, and so has mapped the arrow keys to scrolling the viewport.

So: "maybe, if you would have used them for something better".

my philosophy is that arrow keys are only bad if you're using them while in insert mode. I find that this was added to appeal to novices, but in the long run, really interferes with learning to use the editor to its fullest.

Only if by accident.