Isolate Cloud Mac CI Jobs with Users, Permissions, and launchctl

When the same cloud Mac handles build jobs back to back, the hardest problems to detect are often not compilation failures. Instead, leftover state from the previous job can make the next one “succeed by accident”: dependencies are already present in the cache, background services still occupy ports, temporary credentials have not been removed, or user-level agents remain active in the GUI session. The pipeline may appear stable in the short term, only to fail repeatedly after moving to another machine or changing the job order. The solution is not to append a blunt deletion command at the end. First define clear boundaries for identities, directories, caches, and session domains, then add verification for each boundary.

Define the boundaries that require isolation

At least five types of state must be checked: the runtime user, working directory, temporary directory, caches, and background processes. The CI service itself may be launched by a system-level daemon, but builds should not routinely run with administrator privileges. A safer approach is to create a standard non-admin user such as ci_runner and use it for checkout, compilation, and testing.

The goal of isolation is not to prevent a job from “seeing the entire machine.” It is to ensure that mutable state created by one job cannot affect the next job unless that state has been explicitly declared.

Record a baseline first so that later investigations have something to compare against:

id
umask
printf 'HOME=%s\nTMPDIR=%s\n' "$HOME" "$TMPDIR"
launchctl print "gui/$(id -u)" >/tmp/launchctl-baseline.txt 2>&1 || true
ps -axo user,pid,ppid,command

If the runner needs to perform a small number of privileged operations, consolidate them into controlled scripts and grant permissions only for those specific actions. Do not run the entire build with elevated privileges. Build scripts should also avoid modifying system-wide toolchains, global network settings, or other users’ directories.

Create a private workspace for every job

Generate each job directory from an unpredictable but auditable job identifier, rejecting slashes, spaces, and control characters. Set directory permissions to 700, and place temporary files inside the job directory so concurrent jobs do not share a system temporary path.

set -eu

case "${RUN_ID:-}" in
  ""|*[!A-Za-z0-9._-]*)
    echo "Invalid RUN_ID" >&2
    exit 64
    ;;
esac

RUN_ROOT="/Users/ci_runner/Jobs/$RUN_ID"
install -d -m 700 "$RUN_ROOT"
install -d -m 700 "$RUN_ROOT/src" "$RUN_ROOT/tmp" "$RUN_ROOT/cache"

export HOME="/Users/ci_runner"
export TMPDIR="$RUN_ROOT/tmp/"
export XDG_CACHE_HOME="$RUN_ROOT/cache"
umask 077

Do not place workspaces in a shared directory writable by every job. If a shared read-only seed cache is necessary, copy it into the local cache at the beginning of the job and allow the build to modify only that copy. This preserves the benefits of a warm cache without allowing a failed job to contaminate the shared baseline.

Isolate with permissions, not naming conventions

Including a job number in the directory name does not make it secure. Use stat -f '%Su %Sp %N' "$RUN_ROOT" to verify ownership and permissions, and ls -lde to inspect additional ACLs. If inherited rules are present, first determine where they came from, then remove unnecessary grants through a controlled initialization process. Do not recursively loosen permissions from within the build script.

Divide caches into three lifetimes

Clearing every cache slows down the pipeline, while reusing everything amplifies contamination. In practice, caches can be divided by lifetime:

At a minimum, cache keys should include the toolchain version, dependency lockfile digest, and target architecture. Using only the branch name allows old artifacts to remain eligible after a toolchain change. After restoring a cache, perform lightweight verification by checking items such as the lockfile digest, architecture of key binaries, and directory ownership. If the copy fails validation, discard it instead of attempting an in-place repair.

Sensitive material does not belong in a cache. Inject short-lived credentials through the job environment and expose them only to the subprocesses that need them. Logs must not print the complete environment. At the end of the job, explicitly unset the relevant variables and verify that generated files were not written to the archive directory.

Understand launchctl session domains correctly

Background agents on macOS may belong to the system domain, user domain, or GUI session domain. Running a build as ci_runner does not automatically mean that every agent it launches is placed in the expected domain. Check the current user identity first, then inspect the target domains:

uid="$(id -u ci_runner)"
sudo -u ci_runner launchctl print "user/$uid" >/tmp/user-domain.txt

if launchctl print "gui/$uid" >/dev/null 2>&1; then
  sudo -u ci_runner launchctl print "gui/$uid" >/tmp/gui-domain.txt
fi

Command-line-only builds usually do not depend on a GUI session. Tests that require a simulator or graphical interface should first verify that a valid session exists. Do not hide a missing session by repeatedly launching agents. Save the PID of every service started temporarily by a job and send a normal termination signal during teardown. Terminating processes in bulk by name can disrupt other jobs on the same machine.

Identify processes left behind across jobs

Capture ps snapshots before and after the job, then correlate processes by user, parent process, and working directory. Checking ports alone is insufficient because file watchers and test daemons without listening ports can still consume resources. Any process that remains after teardown and whose command line points to the current RUN_ROOT should be treated as a failure rather than silently ignored.

Add exit verification that can fail

The cleanup phase must preserve the original job exit code while still making cleanup errors observable. Run the checks in a fixed order:

1. Stop subprocesses started by the job and wait for them to exit.
2. Check the workspace for sockets, mount points, or files with unexpected permissions.
3. Copy test reports to a controlled archive directory outside the workspace.
4. Remove temporary job credentials and environment variables.
5. Validate the path prefix, then delete the job directory.
6. Capture process and launchctl state again and compare the results with the baseline.

Always protect the path before deletion:

cleanup() {
  case "$RUN_ROOT" in
    /Users/ci_runner/Jobs/*)
      rm -rf -- "$RUN_ROOT"
      ;;
    *)
      echo "Refusing unsafe cleanup path" >&2
      return 1
      ;;
  esac
}
trap cleanup EXIT HUP INT TERM

The verification script should not suppress every error merely to keep the pipeline green. Archive failures, leftover processes, and changes in directory ownership should all produce an explicit nonzero status. For continuously running CI on SoarMac, first confirm the currently available configurations in the console, then decide whether to split runners according to the memory and disk pressure created by concurrent jobs. Regardless of the selected configuration, identity and state boundaries should remain consistent.

Pre-deployment checklist

During initial integration, run two jobs with different content but identical steps back to back. Deliberately make the first job create a cache file, a background process, and a temporary variable. The second job must be unable to read undeclared files or inherit the previous job’s processes and credentials. Finally, verify the following:

• The runner uses a dedicated non-admin user;
• Every job directory has 700 permissions, and its path passes format validation;
• Temporary directories and writable caches are not shared across jobs by default;
• Machine-scoped seed caches are read-only for the build user;
• GUI tests explicitly identify the launchctl session domain they depend on;
• Subprocesses are stopped by PID or job relationship, not killed indiscriminately by name;
• Deletion occurs only after archiving is complete;
• Cleanup failures cause the job to fail instead of producing only a log message.

Once these checks can be repeated reliably, pipeline success comes from declared inputs rather than whatever state happens to remain on the machine. Whether concurrency is adjusted, jobs are migrated, or runtime directories are changed later, the same boundaries provide a fast way to identify where differences originate.

Frequently asked questions