Handle underscores

Speed up module parsing with hand-written parser

Trivial: Simplify ensureDirectory ...

mkdirRecurse will not do anything if the directory already exists,
so there is no need to call existsDirectory.

Fix ubyte[] download overload by avoiding a cast ...

We were using the default template parameter, `char`,
and as the documentation for `get` states:
> If asking for char, content will be converted from
> the connection character set (specified in HTTP
> response headers or FTP connection properties,
> both ISO-8859-1 by default) to UTF-8.

However we are downloading a binary so converting
it to UTF-8 will fail. Especially if we intend
to cast it to `ubyte[]` afterwards.

Trivial: Print the Exception message in retryDownload ...

This makes the source of the issue more obvious when using
the verbose flag.

Remove FileInfo.timeCreated ...

As it is not available on all platforms, we can't really rely
on it for code, and it is indeed not used anywhere.

Introduce readText and make stripUTF8BOM private

Tests: Fix writeFile implementation with vibe-d ...

This was triggering an assert error with vibe-d path.
The recursion logic was a bit murky, so the function
is now split in 3 different part and better documented.

Make all overloads of download private ...

Since this is not used directly anywhere inside of dub,
make it private so it can be later refactored.

Update configy to the latest HEAD

CI: Move single checks to Github, remove dead code ...

Remove support for very old frontend - we were not testing it anyways.
Move the useful checks to another job so they run once.

Improve unittest framework to have a virtual FS ...

This gets the unittest framework closer to the actual behavior,
and allows us to mock basic FS operations such as mkdir and writeFile.
Further improvements can be made to support more operations,
and to have package setup done before `TestDub` instantiation.

Configy: Synchronize with upstream ...

Bring in some documentation improvements and PR48 bugfix.

Build with -preview=in ...

This reduces the number of deprecations issued in dub.

Trivial: Inline unused listDirectory into iterateDirectory ...

In practice we only every iterate directories in a foreach,
which means iterateDirectory is the only function ever used.

Trivial: Remove unused listDirectory string overload

Remove unused iterateDirectory(string) ...

It was actually unused, because al the call sites were doing a useless
round-trip from NativePath to string to NativePath.
There is no need for it anymore so we can just remove it to avoid
this kind of mistakes in the future.

Don't wastefully execute everything via a shell ...

We should only ever use the shell if we want to use the shell's
features, such as pipes, non-trivial redirection, or shell language
features (branches/loops/etc.).  Otherwise, executing a program via
the shell is pointless and wasteful.

Add unittest to test collection of C sources and headers

Make getDUBExePath returns a NativePath

review adjustments

fix typo(s)

Merge remote-tracking branch 'upstream/stable' into merge_stable

reduce dependency on the working directory ...

Improves library API usage and bugs with the build generator

fix cyclic dependency of dub.internal.utils

replace std.variant with std.sumtype (#2550) ...

Variant is quite big and runtime heavy, with sumtype we already cover
wrong type cases at compile time.

Fix various DDOC warnings

Revert "Remove Vibe.d HTTP support / library-nonet" ...

This reverts commit db49cbe.

Replace all instances of mkdirRecurse with ensureDir ...

All usages actually want to ensure the directory is present,
and don't care about having a separation between existence check
and creation.

Move foreign code from global scope to dub.internal. ...

Avoids conflicting with the actual dub packages when working with dub as a library. Also, even if just by convention, it ensures that dependent code doesn't rely on this code as a public API.