-
-
Notifications
You must be signed in to change notification settings - Fork 34.5k
test_runner: add experimental mock.fs API
#61468
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Review requested:
|
| * @enum {('readFile'|'writeFile'|'appendFile'|'stat'|'lstat'|'access'| | ||
| * 'exists'|'unlink'|'mkdir'|'rmdir'|'readdir')[]} Supported fs APIs | ||
| */ | ||
| const SUPPORTED_APIS = [ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what are the unsupported ones? is the plan to support them gradually?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rename, copyFile, rm, symlinks, etc. There is a lot of other operations from fs that arent included in here. And yes, I plan to support all of them gradually, although I think the operations in this iteration looks fine for the initial scope
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #61468 +/- ##
==========================================
+ Coverage 88.54% 89.83% +1.28%
==========================================
Files 704 672 -32
Lines 208753 204401 -4352
Branches 40280 39272 -1008
==========================================
- Hits 184847 183621 -1226
+ Misses 15907 13125 -2782
+ Partials 7999 7655 -344
🚀 New features to boost your workflow:
|
|
Thanks for the contribution! I will take a look ASAP, though I think it would be great to have a review from @nodejs/fs as well. I'm also wondering if it might make sense to break down this PR into even more granular iterations, as it's already a non-trivial change! @nodejs/test_runner any thoughts? |
|
Thanks @ljharb for the comment, really appreciate it! There was a lot of interesting stuff I learned while reviewing your questions. Some of them should already be resolved, while I'll resolve/comment on the others ASAP. @pmarchini thanks and I think that makes total sense. In issue #55902, we already exchanged some ideas about this. I think it makes sense to centralize that discussion there, what do you think? Either way, I'm happy to close this PR and work it based on the chosen scope and granularity, if necessary! |
|
The
notable-change
Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. Otherwise, the commit will be placed in the Other Notable Changes section. |
|
|
||
| Enables file system mocking. | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should go after the typed list
| **Note:** When file system mocking is enabled, the mock automatically | ||
| creates parent directories for all virtual files. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| **Note:** When file system mocking is enabled, the mock automatically | |
| creates parent directories for all virtual files. | |
| When file system mocking is enabled, the mock automatically | |
| creates parent directories for all virtual files. |
| added: REPLACEME | ||
| --> | ||
|
|
||
| > Stability: 1.0 - Early development |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| > Stability: 1.0 - Early development | |
| > Stability: 1.0 - Early development. Enable this API via ... |
| | Property | Type | Default | | ||
| | ------------------------------------------- | -------- | -------------------------------- | | ||
| | `dev` | `number` | `0` | | ||
| | `ino` | `number` | `0` | | ||
| | `mode` | `number` | File: `0o100644`, Dir: `0o40644` | | ||
| | `nlink` | `number` | `1` | | ||
| | `uid` | `number` | `0` | | ||
| | `gid` | `number` | `0` | | ||
| | `rdev` | `number` | `0` | | ||
| | `size` | `number` | Content length | | ||
| | `blksize` | `number` | `4096` | | ||
| | `blocks` | `number` | `ceil(size / 512)` | | ||
| | `atime`/`mtime`/`ctime`/`birthtime` | `Date` | Creation time | | ||
| | `atimeMs`/`mtimeMs`/`ctimeMs`/`birthtimeMs` | `number` | Creation time (ms) | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder, and perhaps this is too much for a single PR, if these should be mock-able.
{
files: {
myFilePath: 'content',
myOtherFile: { atime: 123, content: 'some data' },
}
}| const { call: FunctionCall, bind: FunctionBind } = Function.prototype; | ||
| const BufferPrototypeToString = FunctionCall.call(FunctionBind, FunctionCall, Buffer.prototype.toString); | ||
| const BufferFrom = Buffer.from; | ||
| const BufferConcat = Buffer.concat; | ||
| const BufferAlloc = Buffer.alloc; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't we have these in primordials?
| const { | ||
| resolve: pathResolve, | ||
| dirname: pathDirname, | ||
| sep: pathSep, | ||
| } = require('path'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| const { | |
| resolve: pathResolve, | |
| dirname: pathDirname, | |
| sep: pathSep, | |
| } = require('path'); | |
| const { | |
| resolve, | |
| dirname, | |
| sep, | |
| } = require('path'); |
OR
| const { | |
| resolve: pathResolve, | |
| dirname: pathDirname, | |
| sep: pathSep, | |
| } = require('path'); | |
| const path = require('path'); |
These renames seem odd to me.
| __proto__: Stats.prototype, | ||
| dev: 0, | ||
| ino: 0, | ||
| mode: isDirectory ? S_IFDIR | mode : S_IFREG | mode, | ||
| nlink: 1, | ||
| uid: 0, | ||
| gid: 0, | ||
| rdev: 0, | ||
| size, | ||
| blksize: DEFAULT_BLKSIZE, | ||
| blocks: MathCeil(size / BLOCK_SIZE), | ||
| atimeMs: nowMs, | ||
| mtimeMs: nowMs, | ||
| ctimeMs: nowMs, | ||
| birthtimeMs: nowMs, | ||
| atime: nowDate, | ||
| mtime: nowDate, | ||
| ctime: nowDate, | ||
| birthtime: nowDate, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we (for some reason) change the structure of this object in fs, we'd need to update it here as well. Can we ensure the validity of this object with a test or get its structure from fs?
| /** | ||
| * @param {string} syscall | ||
| * @param {string} filepath | ||
| * @returns {Error} | ||
| */ | ||
| function createENOENT(syscall, filepath) { | ||
| return new UVException({ | ||
| __proto__: null, | ||
| errno: UV_ENOENT, | ||
| syscall, | ||
| path: filepath, | ||
| message: 'no such file or directory', | ||
| }); | ||
| } | ||
|
|
||
| /** | ||
| * @param {string} syscall | ||
| * @param {string} filepath | ||
| * @returns {Error} | ||
| */ | ||
| function createENOTDIR(syscall, filepath) { | ||
| return new UVException({ | ||
| __proto__: null, | ||
| errno: UV_ENOTDIR, | ||
| syscall, | ||
| path: filepath, | ||
| message: 'not a directory', | ||
| }); | ||
| } | ||
|
|
||
| /** | ||
| * @param {string} syscall | ||
| * @param {string} filepath | ||
| * @returns {Error} | ||
| */ | ||
| function createENOTEMPTY(syscall, filepath) { | ||
| return new UVException({ | ||
| __proto__: null, | ||
| errno: UV_ENOTEMPTY, | ||
| syscall, | ||
| path: filepath, | ||
| message: 'directory not empty', | ||
| }); | ||
| } | ||
|
|
||
| /** | ||
| * @param {string} syscall | ||
| * @param {string} filepath | ||
| * @returns {Error} | ||
| */ | ||
| function createEEXIST(syscall, filepath) { | ||
| return new UVException({ | ||
| __proto__: null, | ||
| errno: UV_EEXIST, | ||
| syscall, | ||
| path: filepath, | ||
| message: 'file already exists', | ||
| }); | ||
| } | ||
|
|
||
| /** | ||
| * @param {string} syscall | ||
| * @param {string} filepath | ||
| * @returns {Error} | ||
| */ | ||
| function createEISDIR(syscall, filepath) { | ||
| return new UVException({ | ||
| __proto__: null, | ||
| errno: UV_EISDIR, | ||
| syscall, | ||
| path: filepath, | ||
| message: 'illegal operation on a directory', | ||
| }); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few notes on these functions:
-
Perhaps I missed some usages, but these functions are only called once, do they need to be functions?
-
We need to ensure that these error messages line up with
fs's. Can we add a test to validate that?
| /** | ||
| * @param {string} base | ||
| * @param {string} name | ||
| * @returns {string} | ||
| */ | ||
| function safePathJoin(base, name) { | ||
| if (StringPrototypeEndsWith(base, pathSep)) { | ||
| return base + name; | ||
| } | ||
| return base + pathSep + name; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's wrong with path.join?
| // Check for dangerous path segments that could lead to prototype pollution. | ||
| const segments = StringPrototypeSplit(filepath, pathSep); | ||
| for (let j = 0; j < segments.length; j++) { | ||
| if (kDangerousPathSegments.has(segments[j])) { | ||
| throw new ERR_INVALID_ARG_VALUE( | ||
| 'options.files', | ||
| filepath, | ||
| 'cannot contain __proto__, constructor, or prototype in path', | ||
| ); | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
iirc, Node.js trusts data passed to it's functions, thus we should trust the data passed here, and not check it for pollution, right?
Add experimental
mock.fsAPI to the test runner for mocking file system operations in tests. This allows tests to simulate file operations without actually reading from or writing to disk.Note: There is an existing draft PR #59194 by @joaoGabriel55 working on this feature, though it appears to have stalled (last updated August 2025). This implementation takes a different approach based on maintainer feedback in the issue discussion:
mock.fsnaming as suggested by @MoLow--experimental-test-fs-mocksflag as suggested by @ljharbFeatures
readFile,writeFile,appendFile,stat,lstat,access,exists,unlink,mkdir,rmdir,readdirisolate: true) to completely block real filesystem accessapisoptionSymbol.disposesupport for automatic cleanupExamples
Basic usage
Isolation mode
Write operations
Async/promises
Directory operations
Next Steps
This PR provides an initial set of commonly-used fs operations. Future iterations could add file operations like
rename,copyFile,rm, andtruncate, as well as symlink support (symlink,readlink,link).Refs: #55902