aboutsummaryrefslogtreecommitdiff
path: root/src/Cache.zig
AgeCommit message (Collapse)Author
2022-07-26std.fmt: require specifier for unwrapping ?T and E!TInKryption
2022-07-24Revert "std.fmt: require specifier for unwrapping ?T and E!T."Andrew Kelley
This reverts commit 7cbd586ace46a8e8cebab660ebca3cfc049305d9. This is causing a fail to build from source: ``` ./lib/std/fmt.zig:492:17: error: cannot format optional without a specifier (i.e. {?} or {any}) @compileError("cannot format optional without a specifier (i.e. {?} or {any})"); ^ ./src/link/MachO/Atom.zig:544:26: note: called from here log.debug(" RELA({s}) @ {x} => %{d} in object({d})", .{ ^ ``` I looked at the code to fix it but none of those args are optionals.
2022-07-24std.fmt: require specifier for unwrapping ?T and E!T.InKryption
Co-authored-by: Veikka Tuominen <git@vexu.eu>
2022-04-22Revert "Fix C include files not being in `whole` cache (#11365)"Andrew Kelley
This reverts commit a430630002bf02162ccbf8d3eb10fd73e490cefd. Wait a minute, I'm sorry, I need to revert this. The whole premise of this change is broken because the point of the hash is that it tells whether the same compilation has been done before. This requires items to be added to the hash in the same sequence every time. This means that introducing a lock is fundamentally broken because the order needs to be the same in future runs of the compiler, and not decided by threads racing against each other. The proper solution to this is to, in whole cache mode, append the hash inputs to some data structure, and then after the compilation is complete, do some kind of sorting on the hash inputs so that they will be the same order every time, then apply them in sequence. No lock on the Cache object is needed for this scheme.
2022-04-22Fix C include files not being in `whole` cache (#11365)Tom Read Cutting
2022-04-18stage2: Only bypass `flock` on WASICody Tapscott
2022-04-18stage2: Bypass file locks in src/Cache.zig for WASI targetsCody Tapscott
2022-01-29fs: Use `OpenMode` enum instead of read/write flags.Sage Hane
2022-01-02stage2: add `@import` and `@embedFile` to CacheHashAndrew Kelley
when using `CacheMode.whole`. Also, I verified that `addDepFilePost` is in fact including the original C source file in addition to the files it depends on.
2021-12-09Cache: use mutex to protect recent_problematic_timestampAndrew Kelley
The previous commit tried to use atomics but not many CPUs support 128-bit atomics. So we use a mutex. In order to avoid contention, we also store `recent_problematic_timestamp` locally on the `Manifest` which is only ever accessed from a single thread at a time, and only consult the global one if the local one is problematic. This commit was tested by running `zig build test-behavior` in two separate terminals at the same time.
2021-12-09Cache: fix data race with is_problematic_timestampAndrew Kelley
Previously `recent_problematic_timestamp` was unprotected and accessed potentially with multiple worker threads simultaneously. This commit protects it with atomics and also introduces a flag to prevent multiple timestamp checks from within the same call to hit(). Unfortunately the compiler-rt function __sync_val_compare_and_swap_16 is not yet implemented, so I will have to take a different strategy in a follow-up commit.
2021-12-09Cache: improvements to previous commitAndrew Kelley
* put `recent_problematic_timestamp` onto `Cache` so that it can be shared by multiple Manifest instances. * make `isProblematicTimestamp` return true on any filesystem error. * save 1 syscall by using truncate=true in createFile instead of calling `setEndPos`.
2021-12-09Cache: fix two issues with isProblematicTimestampTravis Martin
1. It was looking for trailing zero bits when it should be looking for trailing decimal zeros. 2. Clock timestamps had more precision than the actual file timestamps The fix is to grab a timestamp from a 'just now changed' temp file. This timestamp is "problematic". Any file timestamp greater than or equal to this timestamp is considered problematic. File timestamps **prior** to this **can** be trusted. Downside is that it causes a disk I/O to write to and then read the timestamp from this file ~1ms on my system. This is partially mitigated by keeping track of the most recent problematic timestamp, and only checking for a new problematic timestamp when checking a timestamp that is equal to or larger than the last problematic one. This fixes #6082.
2021-11-30allocgate: std Allocator interface refactorLee Cannon
2021-11-24Cache: add debug log statementAndrew Kelley
2021-11-20stage2: default dynamic libraries to be linked as neededAndrew Kelley
After this change, the default for dynamic libraries (`-l` or `--library`) is to only link them if they end up being actually used. With the Zig CLI, the new options `-needed-l` or `--needed-library` can be used to force link against a dynamic library. With `zig cc`, this behavior can be overridden with `-Wl,--no-as-needed` (and restored with `-Wl,--as-needed`). Closes #10164
2021-10-04migrate from `std.Target.current` to `@import("builtin").target`Andrew Kelley
closes #9388 closes #9321
2021-09-19Update all ensureCapacity calls to the relevant non-deprecated versionRyan Liptak
2021-08-06Update all usages of mem.split/mem.tokenize for generic versionRyan Liptak
2021-06-29std: implement a cross platform file locking abstractionAndrew Kelley
This modifies the lock semantics from using AccessMode to using NtLockFile/NtUnlockFile. This is a breaking change.
2021-06-29stage2: Cache system handles shared objectsAndrew Kelley
Fixes #9139 Fixes #9187
2021-06-03Breaking hash map changes for 0.8.0Martin Wickham
- hash/eql functions moved into a Context object - *Context functions pass an explicit context - *Adapted functions pass specialized keys and contexts - new getPtr() function returns a pointer to value - remove functions renamed to fetchRemove - new remove functions return bool - removeAssertDiscard deleted, use assert(remove(...)) instead - Keys and values are stored in separate arrays - Entry is now {*K, *V}, the new KV is {K, V} - BufSet/BufMap functions renamed to match other set/map types - fixed iterating-while-modifying bug in src/link/C.zig
2021-05-12translate-c: Ensure extra_cflags are passed to clangEvan Haas
Additionally ensure that the Zig cache incorporates any extra cflags when using translate-c. Fixes the issue identified in #8662
2021-05-08update usage of std.testing in stage2Veikka Tuominen
2021-03-01std: Deprecate 'x'/'X'/'e'/'E' special cases for u8 slicesLemonBoy
Let's follow the road paved by the removal of 'z'/'Z', the Formatter pattern is nice enough to let us remove the remaining four special cases and declare u8 slices free from any special casing!
2021-02-21std: Add more error checking in hexToBytesLemonBoy
Prevent the function from turning into an endless loop that may or may not perform OOB accesses.
2021-01-07Reduce use of deprecated IO typesJay Petacat
Related: #4917
2021-01-03stage2: remove the Cache deadlock detection codeAndrew Kelley
It's more trouble than it's worth; it didn't even catch the most recent incident because it was across process boundaries anyway.
2021-01-03stage2: fix Cache debug deadlock code memory leakAndrew Kelley
2021-01-03stage2: enable building test-stage2 with LLVM backend enabledTimon Kruiper
We can now run `zig build test-stage2 -Denable-llvm`.
2021-01-02stage2: Use {s} instead of {} when formatting stringsLemonBoy
2020-12-28stage2: better error message for root zig source file not foundAndrew Kelley
closes #6777 closes #6893
2020-12-26stage2: Cache deadlock debugging only for safe build modesAndrew Kelley
2020-12-25stage2: Cache: fix resource management of the deadlock debug codeAndrew Kelley
2020-12-25stage2: Cache: add debug deadlock detection codeAndrew Kelley
2020-12-11stage2: detect redundant C/C++ source filesAndrew Kelley
Cache exposes BinDigest. Compilation gains a set of a BinDigest for every C/C++ source file. We detect when the same source/flags have already been added and emit a compile error. This prevents a deadlock in the caching system. Closes #7308
2020-12-01stage2: add -femit-foo=bar args to the cache hashAndrew Kelley
Closes #6979 Closes #7036
2020-10-23When checking a cache hit, make sure to handle a (re)moved source fileTimon Kruiper
When a file cannot be found in the cache, it is not a hit. Closes #6729
2020-10-17std/crypto: make the whole APIs more consistentFrank Denis
- use `PascalCase` for all types. So, AES256GCM is now Aes256Gcm. - consistently use `_length` instead of mixing `_size` and `_length` for the constants we expose - Use `minimum_key_length` when it represents an actual minimum length. Otherwise, use `key_length`. - Require output buffers (for ciphertexts, macs, hashes) to be of the right size, not at least of that size in some functions, and the exact size elsewhere. - Use a `_bits` suffix instead of `_length` when a size is represented as a number of bits to avoid confusion. - Functions returning a constant-sized slice are now defined as a slice instead of a pointer + a runtime assertion. This is the case for most hash functions. - Use `camelCase` for all functions instead of `snake_case`. No functional changes, but these are breaking API changes.
2020-10-09fixups to previous commitAndrew Kelley
* std.fs.Dir.readFile: add doc comments to explain what it means when the returned slice has the same length as the supplied buffer. * introduce readSmallFile / writeSmallFile to abstract over the decision to use symlink or file contents to store data.
2020-09-28stage2: fix Cache not calling ftruncate in writeManifestAndrew Kelley
this led to a corrupt cache when the number of files got smaller. it is now fixed.
2020-09-28stage2: building mingw-w64 and COFF LDD linkingAndrew Kelley
still TODO is the task of creating import .lib files for DLLs on the fly both for -lfoo and for e.g. `extern "kernel32"`
2020-09-24stage2: implement @cImportAndrew Kelley
Also rename Cache.CacheHash to Cache.Manifest
2020-09-22stage2: implement zig buildAndrew Kelley
As part of this: * add std.process.cleanExit. closes #6395 - use it in several places * adjust the alignment of text in `zig build --help` menu * Cache: support the concept of "unhit" so that we properly keep track of the cache when we find out using the secondary hash that the cache "hit" was actually a miss. Use this to fix false negatives of caching of stage1 build artifacts. * fix not deleting the symlink hash for stage1 build artifacts causing false positives. * implement support for Package arguments in stage1 build artifacts * update and add missing usage text * add --override-lib-dir and --enable-cache CLI options - `--enable-cache` takes the place of `--cache on` * CLI supports -femit-bin=foo combined with --enable-cache to do an "update file" operation. --enable-cache without that argument will build the output into a cache directory and then print the path to stdout (matching master branch behavior). * errors surfacing from main() now print "error: Foo" instead of "error: error.Foo".
2020-09-21rename src-self-hosted/ to src/Andrew Kelley