This was not currently tested explicitly, and is currently
the reason why we still have the scanning behavior in the
PackageManager.
In order to remove the scanning behavior and have the PM
lazily scan packages, we first must add test to ensure
there will be no further regression.