How to Reproduce Research Environments with conda-lock on Apple Silicon Mac and Linux HPC? 2026

Lock osx-arm64 and the Linux platform your cluster actually uses as separate targets, install each environment on its own system, then test the same representative workflow against your project baseline. conda-lock supports resolving lock files for multiple declared platforms, but that does not make the resulting binaries interchangeable or prove that either installation can run your complete research project.

This runbook is for researchers who develop or pre-validate on an Apple Silicon Mac before submitting work to Linux HPC.
It also helps students hand an analysis environment to a research group, and technical staff distinguish a successful install from a validated workflow.

01 Stage 1: Define what runs on the Mac and what runs on HPC

Start by splitting the workflow into work that benefits from a macOS development environment and work that belongs on the university cluster. An Apple Silicon Mac can be useful for editing, interactive analysis, and pre-validating a project. Keep batch jobs or resource-intensive computation on the HPC system, following the institution’s compute policy.

Use one actual project as the test case. Write down its inputs, processing steps, and outputs before changing its environment. For example, identify the input files, the command or script that starts the analysis, intermediate files that matter, and the final output you will compare. This becomes the acceptance baseline for both systems.

Do not start by replacing the only environment that currently works. Preserve its specification and record how it is activated. If you already have an environment, Conda’s environment management guidance describes how to inspect and manage environments. Conda’s environment export documentation can also help you capture environment information, but treat an export as input to review—not as proof that the environment is portable.

Before you lock dependencies, inventory the parts Conda may not own:

  • Packages listed in environment.yml, including channel declarations.
  • Any pip section, private package source, or project-local package.
  • Compilers, system libraries, cluster modules, and external executables.
  • License files, credentials, access tokens, and data mounted by the institution.
  • Hardware or filesystem assumptions embedded in scripts.

A cluster module or external system library is not automatically a Conda package just because a project needs it. Keep those requirements visible in a separate setup note. That prevents a lock file from being mistaken for a complete machine image.

02 Stage 2: Declare shared requirements and platform differences

Use the project specification to describe intended top-level dependencies. Then resolve the platforms separately. For an Apple Silicon Mac, the target is osx-arm64; use the Linux target that matches the cluster rather than assuming that every HPC system has the same architecture. The Conda platform names and conda-lock’s platform options are documented in its basic usage and command-line reference.

A common command pattern is:

conda-lock -f environment.yml \
  -p osx-arm64 \
  -p linux-64

Use linux-64 only when it matches the cluster you intend to target. Check the system information and the university’s documentation before choosing it. The conda-lock command options explain supported flags and configuration. Keep the command, tool version, channels, input specification, and generated lock file together in the project history.

Environment target What to lock What to verify outside the lock
Apple Silicon Mac: osx-arm64 Packages resolved for the Mac platform Mac-compatible package builds, system tools, local data paths
Linux HPC: cluster platform, such as linux-64 when applicable Packages resolved for the cluster target Cluster policy, modules, compute-node access, external libraries
Both targets Shared top-level project requirements and platform-specific resolution Workflow behavior, output checks, licenses, credentials, and data access

The point is not to force both systems to install identical binaries. A platform-specific build may differ even when the top-level dependency intent is shared. If a package is unavailable for one target, decide whether the project can use a supported alternative, whether that step must remain on one platform, or whether the dependency specification needs to change. Do not silently remove a dependency to make a lock command pass.

Keep platform conditions explicit. A lock file can describe dependency resolution for declared platforms, but it cannot guarantee that every package, channel, compiler, license, or institution-managed resource exists on both sides. If a dependency has a platform restriction, record why it exists and which workflow step relies on it. Review the conda-lock output and installation guidance before choosing how the generated file will be consumed.

03 Stage 3: Create an isolated Mac validation environment

Generate or update the lock file from the reviewed specification. Then create a separate environment for validation; do not overwrite the environment used for current research work. The conda-lock installation instructions describe how to install from its generated lock file. Follow the syntax documented for the conda-lock version you have selected.

After installation, collect evidence rather than relying on a successful command exit:

  • Confirm the environment is active and the expected platform was selected.
  • Inspect installed packages and their sources.
  • Import the project’s core dependencies.
  • Start the project’s actual entry point using a small, non-sensitive test input.
  • Save the command, relevant output, and any installation error.

If installation fails, isolate the cause before changing the project specification. Check whether the package exists on the configured channel for osx-arm64, whether the declared constraints conflict, and whether the failure comes from a system component outside Conda. A failed lock resolution, a failed package installation, and a failed project run are different events. Record which one occurred.

Use a project copy or a clean working tree for the test. This lets you compare changes without disturbing ongoing analysis. If the Mac test needs a private package source, provide access through an approved mechanism. Do not place access credentials in the lock file or commit them with the project.

A lock file is evidence of a resolved dependency plan for its declared targets. It is not evidence that the target machine can reach every package source, satisfy every external prerequisite, or complete the research workflow.

04 Stage 4: Install and check the cluster environment

Use the Linux lock target only after confirming it matches the actual cluster platform. Follow your institution’s rules for package installation, network access, software modules, and where computation is allowed. Some clusters restrict outbound access or package installation on login nodes. If so, use the approved transfer or installation route rather than trying to bypass the policy.

Separate the cluster’s setup into two records:

  • Lock-managed dependencies: Conda and any pip requirements represented in the environment definition.
  • Cluster-managed prerequisites: modules, system libraries, licenses, filesystem mounts, and other requirements supplied or governed by the institution.

Check whether environment creation needs internet access, a preloaded package cache, administrator action, or a module setup step. Ask the cluster support team when documentation does not settle the question. A lock file cannot grant permissions or make a restricted package source reachable.

Do not treat a login-node test as acceptance for a compute job. The job may run under a different shell, module state, working directory, scheduler allocation, or filesystem access. Test using the institution’s approved compute workflow and capture the job script, environment activation steps, and logs. If the environment installs but the job fails, preserve the failure details before editing dependencies.

For pip-managed packages, inspect exactly how each package is obtained and installed. The conda-lock pip documentation explains its handling of pip dependencies. Private indexes, local paths, editable checkouts, and credentials can create conditions that are not reproduced simply by sharing a lock file. Document those conditions separately and verify them under the cluster’s approved access method.

FAQ: portability and lock-file boundaries

Can you move a Linux HPC Conda environment straight to an Apple Silicon Mac?
Do not treat a Linux environment as a portable set of binaries. Use its declared dependencies as a starting point, resolve the Mac target separately, and check package availability and system prerequisites on macOS. Then run the project’s representative test. A successful export or Mac installation alone does not establish that the research workflow has transferred correctly.

How do you generate different Mac and Linux lock targets for one project?
Keep the shared dependency intent in the project specification, then resolve the Mac and the cluster’s actual Linux platform as separate targets. Record the channels, input file, and conda-lock version alongside the result. A platform lock records a resolution plan; you still need to install it on the real target system and check institutional restrictions.

Does a successful installation on both systems prove that research results match?
No. Installation confirms that dependency creation completed, not that the project produced an acceptable scientific result. Run the same representative task with traceable inputs on both systems, then compare outputs using the project’s approved baseline and tolerances. Document any difference rather than assuming a universal numerical equality rule.

Will a conda-lock file capture every pip-installed research dependency?
Pip requirements can be included in an environment definition, but inspect their sources and installation conditions. A private index, local project path, editable checkout, or credential requirement may need separate documentation and access handling. Consult the pip-specific guidance, then verify the installation on each target rather than inferring coverage from the presence of a lock file.

05 Stage 5: Run a representative task and review the baseline

With both environments installed, run the same small, representative project task on the Mac and HPC. Use the same documented input files where permitted, the same project revision, and the intended entry point. Keep the test small enough to inspect, but representative enough to exercise the dependencies and processing path that matter.

Record the following for each run:

  • Project revision and input identifiers.
  • Environment file and platform lock target.
  • Activation and execution commands.
  • Relevant logs, errors, and generated outputs.
  • The project’s expected result or approved comparison method.

Compare outputs against the baseline defined by the research project, not against an invented universal threshold. The appropriate tolerance depends on the analysis and its accepted validation method. If results differ, investigate sources such as random processes, parallel execution, platform-specific numerical libraries, input handling, and external system components. Record the explanation and the decision made by the project owner.

Keep these outcomes separate:

  • Environment created: package installation completed.
  • Workflow completed: the project ran through its intended path.
  • Result reviewed: outputs were assessed against the project’s approved standard.

Do not report the first outcome as proof of the third. If the Mac passes a pre-validation task but the cluster job fails, the Mac result has still served a useful diagnostic role; it has not accepted the HPC workflow.

06 Stage 6: Hand off the environment and define when to rebuild

Put the platform lock files, reviewed environment specification, dependency-source notes, setup instructions, and test record under project version control. Keep secrets out of that record. Store credentials through an institution-approved process, and document where authorized users obtain access without embedding private keys or passwords in the project.

Write down the conditions that trigger a new resolution and another review. Examples include changing top-level dependencies, changing channels, updating the lock tool, changing the cluster platform or module setup, and discovering that a package source has changed. When any of these conditions apply, regenerate the relevant platform lock and rerun the representative task on that platform.

Use this handoff checklist before another group member takes over:

  • [ ] The working environment and project baseline are preserved.
  • [ ] The project specification lists intended dependencies and channels.
  • [ ] Separate lock targets match the Mac and the cluster platforms.
  • [ ] Tool version and lock-generation instructions are recorded.
  • [ ] System libraries, modules, licenses, and other external requirements are documented.
  • [ ] Private package access is handled without committing credentials.
  • [ ] Installation evidence is recorded separately for each platform.
  • [ ] The same representative task has been run on the Mac and HPC.
  • [ ] Outputs have been checked against the project’s approved baseline.
  • [ ] Rebuild and revalidation triggers are clear to the next maintainer.

If your group needs an Apple Silicon Mac only for environment setup or pre-validation, a remotely accessed Mac can fill that temporary role without replacing the university cluster. Check the CALMVPS remote Mac access options and current plan details before deciding. This is a fit when you need a real macOS environment but do not have local Mac hardware available. It is not a substitute for HPC when your job needs cluster-scale resources, and it is not the right choice if your workflow requires physical equipment attached to a local Mac. Keep the cluster as the execution target, use the Mac to validate its own platform-specific steps, and retain the separate lock files and test evidence as the handoff record.