Caching and pre-filtering parents avoids some of the worst O(n^2) loops.
Adds a basic test to check for minimum performance and scaling. We'll
have to see if our test machine handles it
Fixes https://gitlab.com/kicad/code/kicad/-/issues/21482
(cherry picked from commit a3eb311c48)