| Age | Commit message (Collapse) | Author |
|---|
|
Functions like isMinGW() and isGnuLibC() have a good reason to exist: They look
at multiple components of the target. But functions like isWasm(), isDarwin(),
isGnu(), etc only exist to save 4-8 characters. I don't think this is a good
enough reason to keep them, especially given that:
* It's not immediately obvious to a reader whether target.isDarwin() means the
same thing as target.os.tag.isDarwin() precisely because isMinGW() and similar
functions *do* look at multiple components.
* It's not clear where we would draw the line. The logical conclusion before
this commit would be to also wrap Arch.isX86(), Os.Tag.isSolarish(),
Abi.isOpenHarmony(), etc... this obviously quickly gets out of hand.
* It's nice to just have a single correct way of doing something.
|
|
|
|
|
|
Fix a bunch of frontend bugs
|
|
Update wasi-libc to d03829489904d38c624f6de9983190f1e5e7c9c5
|
|
Clearing the analysis roots was very clever and all, but not actually
valid. We need to avoid *any* reference to the analysis errors if there
were any fatal files, and that includes sorting the errors!
Resolves: #22774
|
|
The changes from a few commits earlier, where semantic analysis no
longer occurs if any Zig files failed to lower to ZIR, mean `file`
dependencies are no longer necessary! However, we now need them for ZON
files, to be invalidated whenever a ZON file changes.
|
|
This came with a big cleanup to `Zcu.PerThread.updateFile` (formerly
`astGenFile`).
Also, change how the cache manifest works for files in the import table.
Instead of being added to the manifest when we call `semaFile` on them,
we iterate the import table after running the AstGen workers and add all
the files to the cache manifest then.
The downside is that this is a bit more eager to include files in the
manifest; in particular, files which are imported but not actually
referenced are now included in analysis. So, for instance, modifying any
standard library file will invalidate all Zig compilations using that
standard library, even if they don't use that file.
The original motivation here was simply that the old logic in `semaFile`
didn't translate nicely to ZON. However, it turns out to actually be
necessary for correctness. Because `@import("foo.zig")` is an
AstGen-level error if `foo.zig` does not exist, we need to invalidate
the cache when an imported but unreferenced file is removed to make sure
this error is triggered when it needs to be.
Resolves: #22746
|
|
This is mainly in preparation for integrating ZonGen into the pipeline
properly, although these names are better because `astGenFile` isn't
*necessarily* running AstGen; it may determine that the current ZIR is
up-to-date, or load cached ZIR.
|
|
|
|
Instead, `source`, `tree`, and `zir` should all be optional. This is
precisely what we're actually trying to model here; and `File` isn't
optimized for memory consumption or serializability anyway, so it's fine
to use a couple of extra bytes on actual optionals here.
|
|
This commit allows using ZON (Zig Object Notation) in a few ways.
* `@import` can be used to load ZON at comptime and convert it to a
normal Zig value. In this case, `@import` must have a result type.
* `std.zon.parse` can be used to parse ZON at runtime, akin to the
parsing logic in `std.json`.
* `std.zon.stringify` can be used to convert arbitrary data structures
to ZON at runtime, again akin to `std.json`.
|
|
|
|
as well as libtsan, libunwind, and libc files
|
|
incremental: handle `@embedFile`
|
|
No longer needed since we disable LTO for mingw32.
|
|
* compiler-rt and mingw32 have both run into LLD bugs, and LLVM disables LTO for
its compiler-rt, so disable LTO for these.
* While we haven't run into any bugs in it, LLVM disables LTO for its libtsan,
so follow suit just to be safe.
* Allow LTO for libfuzzer as LLVM does.
|
|
Uses of `@embedFile` register dependencies on the corresponding
`Zcu.EmbedFile`. At the start of every update, we iterate all embedded
files and update them if necessary, and invalidate the dependencies if
they changed.
In order to properly integrate with the lazy analysis model, failed
embed files are now reported by the `AnalUnit` which actually used
`@embedFile`; the filesystem error is stored in the `Zcu.EmbedFile`.
An incremental test is added covering incremental updates to embedded
files, and I have verified locally that dependency invalidation is
working correctly.
|
|
and remove faulty assertion. When a prelink task fails, the
completed_prelink_tasks counter will not decrement.
A future improvement will be needed to make the pipeline fully robust
and handle failed prelink tasks, followed by updates in which those
tasks succeed, and compilation proceeds like normal.
Currently if a prelink task fails, the Compilation will be left in a
state unrecoverable by an incremental update.
|
|
This moves the default value logic to Package.Module.create() instead and makes
it so that Compilation.Config.any_unwind_tables is computed similarly to
any_sanitize_thread, any_fuzz, etc. It turns out that for any_unwind_tables, we
only actually care if unwind tables are enabled at all, not at what level.
|
|
See: https://github.com/WebAssembly/tool-conventions/pull/235
This is not *quite* using the same features as the spec'd lime1 model because
LLVM 19 doesn't have the level of feature granularity that we need for that.
This will be fixed once we upgrade to LLVM 20.
Part of #21818.
|
|
Turns out that even modern Debian aarch64 glibc libc_nonshared.a has
references to _init, meaning that the previous commit caused a
regression when trying to build any -lc executable on that target.
This commit backs out the changes to LibCInstallation.
There is still a fork in the road coming up when the self-hosted ELF
linker becomes load bearing on that target.
|
|
crti.o/crtn.o is a legacy strategy for calling constructor functions
upon object loading that has been superseded by the
init_array/fini_array mechanism.
Zig code depends on neither, since the language intentionally has no way
to initialize data at runtime, but alas the Zig linker still must
support this feature since popular languages depend on it.
Anyway, the way it works is that crti.o has the machine code prelude of
two functions called _init and _fini, each in their own section with the
respective name. crtn.o has the machine code instructions comprising the
exitlude for each function. In between, objects use the .init and .fini
link section to populate the function body.
This function is then expected to be called upon object initialization
and deinitialization.
This mechanism is depended on by libc, for example musl and glibc, but
only for older ISAs. By the time the libcs gained support for newer
ISAs, they had moved on to the init_array/fini_array mechanism instead.
For the Zig linker, we are trying to move the linker towards
order-independent objects which is incompatible with the legacy
crti/crtn mechanism.
Therefore, this commit drops support entirely for crti/crtn mechanism,
which is necessary since the other commits in this branch make it
nondeterministic in which order the libc objects and the other link
inputs are sent to the linker.
The linker is still expected to produce a deterministic output, however,
by ignoring object input order for the purposes of symbol resolution.
|
|
|
|
Move all the remaining Jobs that produce linker inputs to be spawned
earlier in the pipeline and registered with link_task_wait_group.
|
|
|
|
This means doing more work in parallel which is already good, but it's
also a correctnes fix because we need link_task_wait_group.wait() to
ensure that no more linker inputs will be generated.
|
|
|
|
|
|
|
|
|
|
yet because they aren't hooked up to the new linker API
|
|
|
|
|
|
|
|
|
|
this strategy uses a "postponed" queue to handle codegen tasks that
spawn too early. there's probably a better way.
|
|
and expose object_host_name as an option for setting the lib name for
object files, since the wasm linking standards don't specify a way to do
it.
|
|
|
|
The goals of this branch are to:
* compile faster when using the wasm linker and backend
* enable saving compiler state by directly copying in-memory linker
state to disk.
* more efficient compiler memory utilization
* introduce integer type safety to wasm linker code
* generate better WebAssembly code
* fully participate in incremental compilation
* do as much work as possible outside of flush(), while continuing to do
linker garbage collection.
* avoid unnecessary heap allocations
* avoid unnecessary indirect function calls
In order to accomplish this goals, this removes the ZigObject
abstraction, as well as Symbol and Atom. These abstractions resulted
in overly generic code, doing unnecessary work, and needless
complications that simply go away by creating a better in-memory data
model and emitting more things lazily.
For example, this makes wasm codegen emit MIR which is then lowered to
wasm code during linking, with optimal function indexes etc, or
relocations are emitted if outputting an object. Previously, this would
always emit relocations, which are fully unnecessary when emitting an
executable, and required all function calls to use the maximum size LEB
encoding.
This branch introduces the concept of the "prelink" phase which occurs
after all object files have been parsed, but before any Zcu updates are
sent to the linker. This allows the linker to fully parse all objects
into a compact memory model, which is guaranteed to be complete when Zcu
code is generated.
This commit is not a complete implementation of all these goals; it is
not even passing semantic analysis.
|
|
|
|
* MSDN documentation page covering what resource IDs manifests should have:
https://learn.microsoft.com/en-us/windows/win32/sbscs/using-side-by-side-assemblies-as-a-resource
* This change ensures shared libraries that embed win32 manifests use the
proper ID of 2 instead of 1, which is only allowed for .exes. If the manifest
uses the wrong ID, it will not be found and is essentially ignored.
|
|
|
|
|
|
This commit reworks how values like the panic handler function are
memoized during a compiler invocation. Previously, the value was
resolved by whichever analysis requested it first, and cached on `Zcu`.
This is problematic for incremental compilation, as after the initial
resolution, no dependencies are marked by users of this memoized state.
This is arguably acceptable for `std.builtin`, but it's definitely not
acceptable for the panic handler/messages, because those can be set by
the user (`std.builtin.Panic` checks `@import("root").Panic`).
So, here we introduce a new kind of `AnalUnit`, called `memoized_state`.
There are 3 such units:
* `.{ .memoized_state = .va_list }` resolves the type `std.builtin.VaList`
* `.{ .memoized_state = .panic }` resolves `std.Panic`
* `.{ .memoized_state = .main }` resolves everything else we want
These units essentially "bundle" the resolution of their corresponding
declarations, storing the results into fields on `Zcu`. This way, when,
for instance, a function wants to call the panic handler, it simply runs
`ensureMemoizedStateResolved`, registering one dependency, and pulls the
values from the `Zcu`. This "bundling" minimizes dependency edges. The 3
units are separated to allow them to act independently: for instance,
the panic handler can use `std.builtin.Type` without triggering a
dependency loop.
|
|
Previously, logic in `Compilation.getAllErrorsAlloc` was corrupting the
`failed_analysis` hashmap. This meant that on updates after the initial
update, attempts to remove entries from this map (because the `AnalUnit`
in question is being re-analyzed) silently failed. This resulted in
compile errors from earlier updates wrongly getting "stuck", i.e. never
being removed.
This commit also adds a few log calls which helped me to find this bug.
|
|
To avoid this PR regressing error messages, most of the work here has
gone towards improving error notes for why code was comptime-evaluated.
ZIR `block_comptime` now stores a "comptime reason", the enum for which
is also used by Sema. There are two types in Sema:
* `ComptimeReason` represents the reason we started evaluating something
at comptime.
* `BlockComptimeReason` represents the reason a given block is evaluated
at comptime; it's either a `ComptimeReason` with an attached source
location, or it's because we're in a function which was called at
comptime (and that function's `Block` should be consulted for the
"parent" reason).
Every `Block` stores a `?BlockComptimeReason`. The old `is_comptime`
field is replaced with a trivial `isComptime()` method which returns
whether that reason is non-`null`.
Lastly, the handling for `block_comptime` has been simplified. It was
previously going through an unnecessary runtime-handling path; now, it
is a trivial sub block exited through a `break_inline` instruction.
Resolves: #22296
|
|
This commit separates semantic analysis of the annotated type vs value
of a global declaration, therefore allowing recursive and mutually
recursive values to be declared.
Every `Nav` which undergoes analysis now has *two* corresponding
`AnalUnit`s: `.{ .nav_val = n }` and `.{ .nav_ty = n }`. The `nav_val`
unit is responsible for *fully resolving* the `Nav`: determining its
value, linksection, addrspace, etc. The `nav_ty` unit, on the other
hand, resolves only the information necessary to construct a *pointer*
to the `Nav`: its type, addrspace, etc. (It does also analyze its
linksection, but that could be moved to `nav_val` I think; it doesn't
make any difference).
Analyzing a `nav_ty` for a declaration with no type annotation will just
mark a dependency on the `nav_val`, analyze it, and finish. Conversely,
analyzing a `nav_val` for a declaration *with* a type annotation will
first mark a dependency on the `nav_ty` and analyze it, using this as
the result type when evaluating the value body.
The `nav_val` and `nav_ty` units always have references to one another:
so, if a `Nav`'s type is referenced, its value implicitly is too, and
vice versa. However, these dependencies are trivial, so, to save memory,
are only known implicitly by logic in `resolveReferences`.
In general, analyzing ZIR `decl_val` will only analyze `nav_ty` of the
corresponding `Nav`. There are two exceptions to this. If the
declaration is an `extern` declaration, then we immediately ensure the
`Nav` value is resolved (which doesn't actually require any more
analysis, since such a declaration has no value body anyway).
Additionally, if the resolved type has type tag `.@"fn"`, we again
immediately resolve the `Nav` value. The latter restriction is in place
for two reasons:
* Functions are special, in that their externs are allowed to trivially
alias; i.e. with a declaration `extern fn foo(...)`, you can write
`const bar = foo;`. This is not allowed for non-function externs, and
it means that function types are the only place where it is possible
for a declaration `Nav` to have a `.@"extern"` value without actually
being declared `extern`. We need to identify this situation
immediately so that the `decl_ref` can create a pointer to the *real*
extern `Nav`, not this alias.
* In certain situations, such as taking a pointer to a `Nav`, Sema needs
to queue analysis of a runtime function if the value is a function. To
do this, the function value needs to be known, so we need to resolve
the value immediately upon `&foo` where `foo` is a function.
This restriction is simple to codify into the eventual language
specification, and doesn't limit the utility of this feature in
practice.
A consequence of this commit is that codegen and linking logic needs to
be more careful when looking at `Nav`s. In general:
* When `updateNav` or `updateFunc` is called, it is safe to assume that
the `Nav` being updated (the owner `Nav` for `updateFunc`) is fully
resolved.
* Any `Nav` whose value is/will be an `@"extern"` or a function is fully
resolved; see `Nav.getExtern` for a helper for a common case here.
* Any other `Nav` may only have its type resolved.
This didn't seem to be too tricky to satisfy in any of the existing
codegen/linker backends.
Resolves: #131
|
|
The `Cau` abstraction originated from noting that one of the two primary
roles of the legacy `Decl` type was to be the subject of comptime
semantic analysis. However, the data stored in `Cau` has always had some
level of redundancy. While preparing for #131, I went to remove that
redundany, and realised that `Cau` now had exactly one field: `owner`.
This led me to conclude that `Cau` is, in fact, an unnecessary level of
abstraction over what are in reality *fundamentally different* kinds of
analysis unit (`AnalUnit`). Types, `Nav` vals, and `comptime`
declarations are all analyzed in different ways, and trying to treat
them as the same thing is counterproductive!
So, these 3 cases are now different alternatives in `AnalUnit`. To avoid
stealing bits from `InternPool`-based IDs, which are already a little
starved for bits due to the sharding datastructures, `AnalUnit` is
expanded to 64 bits (30 of which are currently unused). This doesn't
impact memory usage too much by default, because we don't store
`AnalUnit`s all too often; however, we do store them a lot under
`-fincremental`, so a non-trivial bump to peak RSS can be observed
there. This will be improved in the future when I made
`InternPool.DepEntry` less memory-inefficient.
`Zcu.PerThread.ensureCauAnalyzed` is split into 3 functions, for each of
the 3 new types of `AnalUnit`. The new logic is much easier to
understand, because it avoids conflating the logic of these
fundamentally different cases.
|
|
|
