| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
This commit makes some big changes to how we track state for Zig source
files. In particular, it changes:
* How `File` tracks its path on-disk
* How AstGen discovers files
* How file-level errors are tracked
* How `builtin.zig` files and modules are created
The original motivation here was to address incremental compilation bugs
with the handling of files, such as #22696. To fix this, a few changes
are necessary.
Just like declarations may become unreferenced on an incremental update,
meaning we suppress analysis errors associated with them, it is also
possible for all imports of a file to be removed on an incremental
update, in which case file-level errors for that file should be
suppressed. As such, after AstGen, the compiler must traverse files
(starting from analysis roots) and discover the set of "live files" for
this update.
Additionally, the compiler's previous handling of retryable file errors
was not very good; the source location the error was reported as was
based only on the first discovered import of that file. This source
location also disappeared on future incremental updates. So, as a part
of the file traversal above, we also need to figure out the source
locations of imports which errors should be reported against.
Another observation I made is that the "file exists in multiple modules"
error was not implemented in a particularly good way (I get to say that
because I wrote it!). It was subject to races, where the order in which
different imports of a file were discovered affects both how errors are
printed, and which module the file is arbitrarily assigned, with the
latter in turn affecting which other files are considered for import.
The thing I realised here is that while the AstGen worker pool is
running, we cannot know for sure which module(s) a file is in; we could
always discover an import later which changes the answer.
So, here's how the AstGen workers have changed. We initially ensure that
`zcu.import_table` contains the root files for all modules in this Zcu,
even if we don't know any imports for them yet. Then, the AstGen
workers do not need to be aware of modules. Instead, they simply ignore
module imports, and only spin off more workers when they see a by-path
import.
During AstGen, we can't use module-root-relative paths, since we don't
know which modules files are in; but we don't want to unnecessarily use
absolute files either, because those are non-portable and can make
`error.NameTooLong` more likely. As such, I have introduced a new
abstraction, `Compilation.Path`. This type is a way of representing a
filesystem path which has a *canonical form*. The path is represented
relative to one of a few special directories: the lib directory, the
global cache directory, or the local cache directory. As a fallback, we
use absolute (or cwd-relative on WASI) paths. This is kind of similar to
`std.Build.Cache.Path` with a pre-defined list of possible
`std.Build.Cache.Directory`, but has stricter canonicalization rules
based on path resolution to make sure deduplicating files works
properly. A `Compilation.Path` can be trivially converted to a
`std.Build.Cache.Path` from a `Compilation`, but is smaller, has a
canonical form, and has a digest which will be consistent across
different compiler processes with the same lib and cache directories
(important when we serialize incremental compilation state in the
future). `Zcu.File` and `Zcu.EmbedFile` both contain a
`Compilation.Path`, which is used to access the file on-disk;
module-relative sub paths are used quite rarely (`EmbedFile` doesn't
even have one now for simplicity).
After the AstGen workers all complete, we know that any file which might
be imported is definitely in `import_table` and up-to-date. So, we
perform a single-threaded graph traversal; similar to what
`resolveReferences` plays for `AnalUnit`s, but for files instead. We
figure out which files are alive, and which module each file is in. If a
file turns out to be in multiple modules, we set a field on `Zcu` to
indicate this error. If a file is in a different module to a prior
update, we set a flag instructing `updateZirRefs` to invalidate all
dependencies on the file. This traversal also discovers "import errors";
these are errors associated with a specific `@import`. With Zig's
current design, there is only one possible error here: "import outside
of module root". This must be identified during this traversal instead
of during AstGen, because it depends on which module the file is in. I
tried also representing "module not found" errors in this same way, but
it turns out to be much more useful to report those in Sema, because of
use cases like optional dependencies where a module import is behind a
comptime-known build option.
For simplicity, `failed_files` now just maps to `?[]u8`, since the
source location is always the whole file. In fact, this allows removing
`LazySrcLoc.Offset.entire_file` completely, slightly simplifying some
error reporting logic. File-level errors are now directly built in the
`std.zig.ErrorBundle.Wip`. If the payload is not `null`, it is the
message for a retryable error (i.e. an error loading the source file),
and will be reported with a "file imported here" note pointing to the
import site discovered during the single-threaded file traversal.
The last piece of fallout here is how `Builtin` works. Rather than
constructing "builtin" modules when creating `Package.Module`s, they are
now constructed on-the-fly by `Zcu`. The map `Zcu.builtin_modules` maps
from digests to `*Package.Module`s. These digests are abstract hashes of
the `Builtin` value; i.e. all of the options which are placed into
"builtin.zig". During the file traversal, we populate `builtin_modules`
as needed, so that when we see this imports in Sema, we just grab the
relevant entry from this map. This eliminates a bunch of awkward state
tracking during construction of the module graph. It's also now clearer
exactly what options the builtin module has, since previously it
inherited some options arbitrarily from the first-created module with
that "builtin" module!
The user-visible effects of this commit are:
* retryable file errors are now consistently reported against the whole
file, with a note pointing to a live import of that file
* some theoretical bugs where imports are wrongly considered distinct
(when the import path moves out of the cwd and then back in) are fixed
* some consistency issues with how file-level errors are reported are
fixed; these errors will now always be printed in the same order
regardless of how the AstGen pass assigns file indices
* incremental updates do not print retryable file errors differently
between updates or depending on file structure/contents
* incremental updates support files changing modules
* incremental updates support files becoming unreferenced
Resolves: #22696
|
|
Closes #21132
According to the XDG Base Directory specification
(https://specifications.freedesktop.org/basedir-spec/latest/#variables),
empty values for these environment variables should be treated the same
as if they are unset. Specifically, for the instances changed in this
commit,
> $XDG_DATA_HOME defines the base directory relative to which
> user-specific data files should be stored. If $XDG_DATA_HOME is either
> not set **or empty**, a default equal to $HOME/.local/share should be
> used.
and
> $XDG_CACHE_HOME defines the base directory relative to which
> user-specific non-essential data files should be stored. If
> $XDG_CACHE_HOME is either not set **or empty**, a default equal to
> $HOME/.cache should be used.
(emphasis mine)
In addition to the case mentioned in the linked issue, all other uses of
XDG environment variables were corrected.
|
|
part of #19063
This is a prerequisite for doing the same for Resinator.
|
|
for lazily built commands such as `zig fmt` and `zig reduce`. Useful if
you want to test a patch to them.
|
|
The INCLUDE variable being used during `.rc` preprocessing was an accidental regression in https://github.com/ziglang/zig/pull/17412.
Closes #17585.
resinator changes:
source_mapping: Protect against NUL bytes in #line filenames
lex: Avoid recalculating column on every tab stop within string literals
Proper error handling for failing to open cwd instead of `catch unreachable`
Use platform-specific delimiter for INCLUDE env var parsing
|
|
Introduce introspect.EnvVar which tracks all the environment variables
that are observed by the compiler, so that we can print them with `zig
env`.
The `zig env` command now prints both the resolved values as well as all
the possibly observed environment variables.
|
|
Instead of checking for absolute paths and current working directories
in various file system operations, there is one simple solution: allow
overriding `std.fs.cwd` on WASI.
os.realpath is back to causing a compile error when used on WASI. This
caused a compile error in the Sema handling of `@src()`. The compiler
should never call realpath, so the commit that made this change is
reverted (95ab942184427e7c9b840d71f4d093931e3e48fb). If this breaks
debug info, a different strategy is needed to solve it other than using
realpath.
I also removed the preopens code and replaced it with something much
simpler. There is no longer any global state in the standard library.
Additionally-
* os.openat no longer does an unnecessary fstat on WASI when O.WRONLY
is not provided.
* os.chdir is back to causing a compile error on WASI.
|
|
In general, we prefer compiler code to use relative paths based on open
directory handles because this is the most portable. However, sometimes
absolute paths are used, and sometimes relative paths are used that go
up a directory.
The recent improvements in 81d2135ca6ebd71b8c121a19957c8fbf7f87125b
regressed the use case when an absolute path is used for the zig lib
directory mixed with a relative path used for the root source file. This
could happen when, for example, running the standard library tests, like
this:
stage3/bin/zig test ../lib/std/std.zig
This happened because the zig lib dir was inferred to be an absolute
directory based on the zig executable directory, while the root source
file was detected as a relative path. There was no common prefix and so
it was not determined that the std.zig file was inside the lib
directory.
This commit adds a function for resolving paths that preserves relative
path names while allowing absolute paths, and converting relative
upwards paths (e.g. "../foo") to absolute paths. This restores the
previous functionality while remaining compatible with systems such as
WASI that cannot deal with absolute paths.
|
|
|
|
This change adds support for locating the Zig executable and the library
and global cache directories, based on looking in the fixed "/zig" and
"/cache" directories.
Since our argv[0] on WASI is just the basename (any absolute/relative
path information is deleted by the runtime), there's very limited
introspection we can do on WASI, so we rely on these fixed directories.
These can be provided on the command-line using `--mapdir`, as follows:
```
wasmtime --mapdir=/cwd::. --mapdir=/cache::"$HOME/.cache/zig" --mapdir=/zig::./zig-out/ ./zig-out/bin/zig.wasm
```
|
|
|
|
closes #9388
closes #9321
|
|
- move ZIG_GLOBAL_CACHE_DIR to `introspect`
- cleanup some dead fields from stage1 codegen
|
|
|
|
|
