Determine why Python 3.12 unit tests are so slow #338

Our Python 3.12 unit tests take roughly ~4x as long (at ~20 mins) to run as our 3.11 tests. I'm not really sure about the reason for this but we should investigate and do what we can to bring the test time down.