Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 9, 2026

Plan to add macOS support to the testsuite workflow

  • Analyze the workflow file and understand the current structure
  • Identify all command-line tools needed by test files (xzcat, sort, cat, mktemp, etc.)
  • Check which sort/cat options are GNU-specific and need replacement
  • Replace GNU long options with portable short options in 13 test files
    • cat --squeeze-blankcat -s (3 eaf tests)
    • sort --ignore-leading-blankssort -b (9 nondominated tests)
    • sort --stablesort -s (1 nondominated test)
  • Add macOS-14 with clang to the matrix in moocore.yml
  • Add brew install xz step for macOS to provide xzcat command
  • Remove FIXME comments from workflow
  • Verify the workflow configuration is correct (YAML syntax validated)
  • Run code review and address all feedback
  • Run security scan (no vulnerabilities found)
  • Add artifact upload for failed test outputs with proper filename handling
  • Fix xzcat not found issue by preserving PATH in test environment
  • Fix mutable default argument anti-pattern in runcmd

Summary

Successfully added macOS-14 support to the testsuite workflow and enhanced debugging capabilities:

  1. Updated 13 test files to use POSIX-compatible short options instead of GNU-specific long options
  2. Added macOS-14 runner with clang to the workflow matrix
  3. Installing xz package via homebrew to provide xzcat on macOS
  4. Added artifact upload for failed test outputs with robust file handling
  5. Fixed environment inheritance in regtest.py to preserve PATH and other environment variables, fixing the "xzcat: command not found" issue on macOS
  6. Fixed mutable default argument anti-pattern in the runcmd function

All changes maintain full backward compatibility with Linux/GNU environments while enabling tests to run on macOS with BSD-based tools.

Original prompt

This section details on the original issue you should resolve

<issue_title>Run the testsuite on macos</issue_title>
<issue_description>See:

# FIXME: needs
# * brew install xz
# * fix options only available in GNU diff
#- { os: macos-13, cc: clang, python-version: '3.10' }
#- { os: macos-14, cc: clang, python-version: '3.10' }
</issue_description>

<agent_instructions>Update the moocore.yml workflow to also run on macos-14 using clang. Do not run on macos-13. Running in macos will require installing any command-line tools required by the *.test files, which are shell scripts, such as xzcat, sort, cat, mktemp, and others. In a few places the testsuite uses command-line options only supported by the GNU version of those tools. regtest.py uses xzdiff and GNU diff in debug mode. This debug mode is not used in the workflow so xzdiff and GNU diff are not needed in macos.</agent_instructions>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI changed the title [WIP] Update workflow to run testsuite on macos-14 using clang Add macOS-14 support to testsuite workflow Jan 9, 2026
Copilot AI requested a review from MLopez-Ibanez January 9, 2026 16:40
@MLopez-Ibanez
Copy link
Contributor

@copilot Two tests fail in macos-14: eafdiff-ALG_1_dat_ALG_2_dat.test and spherical-3d-2000pts-10.test. Currently, the workflow does not upload the output of failed tests as artifacts. The output of failed tests is saved in the default tmpdir of the runner and have extension .out or .out.xz. Modify the workflow to do upload such files as artifacts if the workflow fails.

Copy link
Contributor Author

Copilot AI commented Jan 9, 2026

@copilot Two tests fail in macos-14: eafdiff-ALG_1_dat_ALG_2_dat.test and spherical-3d-2000pts-10.test. Currently, the workflow does not upload the output of failed tests as artifacts. The output of f...

Added workflow steps to collect and upload failed test outputs as artifacts when tests fail. The workflow now:

  1. Detects the system's temp directory using Python's tempfile module
  2. Finds all *_*.out and *_*.out.xz files in the temp directory
  3. Uploads them as artifacts with the name pattern failed-tests-{os}-{compiler}

This will help debug the two failing tests (eafdiff-ALG_1_dat_ALG_2_dat.test and spherical-3d-2000pts-10.test) on macOS-14.

Changes in commit d5e9571.

Copilot AI changed the title Add macOS-14 support to testsuite workflow Add macOS-14 support and artifact upload to testsuite workflow Jan 9, 2026
@MLopez-Ibanez MLopez-Ibanez force-pushed the copilot/run-testsuite-on-macos-14 branch from bfb6119 to 1d7f3a9 Compare January 9, 2026 17:14
@MLopez-Ibanez
Copy link
Contributor

@copilot The failed output of eafdiff-ALG_1_dat_ALG_2_dat.test says:

./eafdiff-ALG_1_dat_ALG_2_dat.test: line 3: xzcat: command not found
./eafdiff-ALG_1_dat_ALG_2_dat.test: line 4: xzcat: command not found

The failed output of spherical-3d-2000pts-10.test says:

./spherical-3d-2000pts-10.test: line 1: xzcat: command not found

So it seems there is a problem executing xzcat in macos-14. Investigate the problem and find a solution.

Copy link
Contributor Author

Copilot AI commented Jan 9, 2026

@copilot The failed output of eafdiff-ALG_1_dat_ALG_2_dat.test says:

./eafdiff-ALG_1_dat_ALG_2_dat.test: line 3: xzcat: command not found
...

The issue was in regtest.py. The runcmd function was creating a new environment dictionary without inheriting PATH from the parent process. When subprocess.run is called with env=dict, it completely replaces the environment instead of updating it, causing xzcat (installed via brew) to not be found.

Fixed by merging the custom environment variables with os.environ.copy() to preserve PATH and all other environment variables while still setting the required test-specific variables.

Changes in commit 6264cab.

@MLopez-Ibanez MLopez-Ibanez force-pushed the copilot/run-testsuite-on-macos-14 branch 2 times, most recently from 908a48d to e2acd64 Compare January 9, 2026 22:23
@MLopez-Ibanez MLopez-Ibanez force-pushed the copilot/run-testsuite-on-macos-14 branch from 14a99a2 to 173eff9 Compare January 9, 2026 22:39
- Replace GNU-specific long options with portable short options
- Update sort --ignore-leading-blanks to sort -b
- Update sort --stable to sort -s
- Update cat --squeeze-blank to cat -s
- Add macOS-14 with clang to workflow matrix
- Add brew install xz step for macOS runners

Co-authored-by: MLopez-Ibanez <2620021+MLopez-Ibanez@users.noreply.github.com>
@MLopez-Ibanez MLopez-Ibanez force-pushed the copilot/run-testsuite-on-macos-14 branch from 2d0fe88 to 9496c0e Compare January 10, 2026 21:42
@MLopez-Ibanez MLopez-Ibanez force-pushed the copilot/run-testsuite-on-macos-14 branch from f423763 to c932ec7 Compare January 13, 2026 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run the testsuite on macos

2 participants