Look up `dub.selections.json` files in parent directories too (#2272) ...

* Look up `dub.selections.json` files in parent directories too

In case the root package directory doesn't contain a `dub.selections.json`
file, look in parent directories too and load the first (deepest) one.

This allows using a 'central' `dub.selections.json` file for a repository
containing multiple dub projects, making it automatically apply to all
builds in that source tree if located in the repository root directory
(unless a local `dub.selections.json` overrides it).

Such an 'inherited' selections file is never mutated when running dub for
a nested project, i.e., changes are always saved to a *local*
`dub.selections.json` file.

Work around file locking dependency in PackageManager ...

The `dub.internal.utils : lockFile` function does not land itself to mocking quite well,
as it exposes a struct containing a `std.stdio : File` instance.
The path of least resistance, for now, is to access `store_` directly from
TestPackageManager by overriding `store`.

Do not use `Package.storeInfo` in `PackageManager.store` ...

`Package.storeInfo` does IO directly which conflicts with our
dependency injection approach. Instead inline the code and
use `this.writeFile` directly.

PackageManager: Make lookup lazy again ...

We want to avoid loading the whole cache whenever lookup is called,
as most invocations of dub should result in a call to lookup,
since dependency resolution is much rarer than building with an
existing dub.selections.json.

Move the I/O part of Project.loadSelections to PackageManager ...

By doing so we can remove a sizeable amount of duplication as well
as the last `Dub.loadPackage` override. The resulting API is not
ideal, for example it doesn't allow to distinguish between parsing
errors and non-existent file, but will have to do for now.

Don't call `getPackageIterator` from `loadGitPackage` ...

`getPackageIterator` API forces us to do a refresh, which loads all
packages on the filesystem. However, for `loadGitPackage`'s use case,
we actually can simplify the check quite a bit.

Doing so means that we can now avoid doing a scan for projects
that use SCM-only dependencies.

Test Plan: On master, running the test (without the fix) triggers:
```
core.exception.AssertError@source/dub/test/base.d(790): Trying to access poisoned path: /dub/user/packages/poison/1.0.0/poison/dub.json
```
While with the fix, the test passes.

Trivial: Remove some no-longer-needed imports ...

Now that we are properly mocking file removal in store,
those imports are not used as the function no longer
does IO directly.

Let unittest use PackageManager.store ...

No need to mock, all filesystem functions have been mocked
by the PackageManager now.

Mock IO in writeLocalPackageList / writeOverrides ...

Mocking IO is required for writeLocalPackageList,
as we want to test the 'add-path' behavior.
While it is not a big concern for overrides, as they
are deprecated and we are unlikely to add new tests,
it is easy to do, and already done for reading them.

Remove the need to mock PackageManager.refresh() ...

We can finally test the full features of the base function,
as we are now mocking the IO, not the logic.

Remove direct IO from Location.scanLocalPackages ...

This is one more step towards removing our mocking of refresh_.

PackageManager: Mock readText, do not mock load ...

This allows us to test the behavior of the base class much better,
by overriding the last IO function, readText.

Move Package.findPackageFile to PackageManager, mock ...

Moving it to PackageManager allows us to call PackageManager.existsFile,
allowing us to mock a lot more things.

Remove the need for intermediate `.zip` file in PackageSupplier ...

By using a different overload of `retryDownload`,
we return the data instead of writing to disk,
ensuring that we don't have a lingering file
in memory and reducing IO.

This follow the same approach for deprecations as previous
`PackageSupplier` deprecations: it's an inevitable, but
easily fixable, breaking change for classes that implement
`PackageSupplier`, but it's a proper deprecation for client
code, based on the assumption that all the classes implementing
`PackageSupplier` are in Dub.

Test: Directly mock writeFile/existsFile from PackageManager ...

This allows us to reduce the number of functions we override,
as we would like to test writeSelections and refresh directly,
not the copy of them we have.

Merge pull request #2866 from Geod24/mlang/Simplify1 ...

Trivial: Simplify PackageManager's loadInternal

Move writing selections file to PackageManager ...

The PackageManager is the main source of IO in Dub,
and having it take responsibility for writing the
selections file reduces the number of special casing
we need to make when doing dependency injection.

Simplify condition.

Trivial: Simplify PackageManager's loadInternal

Properly fix issue #2691 by reverting to the original scanning behavior. ...

Reverts to the scanning/warning behavior that was in place before the warnings for the new package directory structure were implemented.

Revert "Fix #2691: Use same structure for managed and unmanaged path" ...

This reverts commit 1a53049.

PackageManager: Move addPackages inside loadGitPackage ...

This avoids calling addPackages when the package is already loaded,
and will allow us to handle PlacementLocation correctly in the future.

Allow better dependency injection on PackageManager's SCM operation ...

What we really want to override is the git clone operation,
so one way to approach this is to decouple it from the rest
of the code, then when our virtual filesystem is complete,
we will only need to override that one function.

Trivial: Slightly simplify PackageManager.store

Merge remote-tracking branch 'upstream/stable' into merge_stable

Do not allow for removal of subpackages ...

Because it can mess up the system quite dramatically.

Improve unittest framework to have a virtual FS ...

This gets the unittest framework closer to the actual behavior,
and allows us to mock basic FS operations such as mkdir and writeFile.
Further improvements can be made to support more operations,
and to have package setup done before `TestDub` instantiation.

Adapt PackageManager.get[Best]Package to PackageName API ...

This paves the way to be able to automatically return subpackages.

Adapt loadSCMPackage to PackageName API

Remove PackageName alias this to toString ...

The alias this was originally added to make the transition
less painful, however due to an overload resolution bug,
it makes it harder at the moment, as thje compiler prefers
calling the deprecated `string` overloads over the
`in PackageName` ones. So remove it and a few hacks along
the way.