Building a conda package requires a recipe. A conda-build recipe is a flat directory that contains the following files:
meta.yaml---A file that contains all the metadata in the recipe. Only
build.sh---The script that installs the files for the package on macOS and Linux. It is executed using the
bld.bat---The build script that installs the files for the package on Windows. It is executed using
run_test.[py,pl,sh,bat]---An optional Python test file, a test script that runs automatically if it is part of the recipe.
Optional patches that are applied to the source.
Other resources that are not included in the source and cannot be generated by the build scripts. Examples are icon files, readme files and build notes.
When you use the conda skeleton command,
the first 3 files---
bld.bat---are automatically generated for you.
Conda-build performs the following steps:
Reads the metadata.
Downloads the source into a cache.
Extracts the source into the source directory.
Applies any patches.
Re-evaluates the metadata, if source is necessary to fill any metadata values.
Creates a build environment and then installs the build dependencies there.
Runs the build script. The current working directory is the source directory with environment variables set. The build script installs into the build environment.
Performs some necessary post-processing steps, such as shebang and rpath.
Creates a conda package containing all the files in the build environment that are new from step 5, along with the necessary conda package metadata.
Tests the new conda package if the recipe includes tests:
Deletes the build environment and source directory to ensure that the new conda package does not inadvertantly depend on artifacts not included in the package.
Creates a test environment with the package and its dependencies.
Runs the test scripts.
The conda-recipes repo contains example recipes for many conda packages.
All recipe files, including
meta.yaml and build
scripts, are included in the final package archive that is
distributed to users. Be careful not to put sensitive information
such as passwords into recipes where it could be made public.
conda skeleton command can help to make
skeleton recipes for common repositories, such as PyPI.
Let's take a closer look at how conda-build uses a recipe to create a package.
When you build a conda package, conda-build renders the package by reading a template in the meta.yaml. See Templating with Jinja.
Templates are filled in using your conda-build config,
which shows the matrix of things to build against. The
conda build config determines how many builds it has to do.
For example, defining a conda_build_config.yaml of the form
and filling it defines a matrix of 4 packages to build:
foo: - 1.0 - 2.0 bar: - 1.2.0 - 1.4.0
After this, conda-build determines what the outputs will be.
For example, if your
conda build config indicates that you
want 2 different versions of Python, conda-build will show
you the rendering for each Python version.
To build the package, conda-build will make an environment for you and install all of the build and run dependencies in that environment. Conda-build will indicate where you can successfully build the package. The prefix will take the form:
<path to conda>/conda-bld/<package name and string>/h_env_placeholder…
Conda-forge downloads your package source and then builds the conda package in the context of the build environment. For example, you may direct it to download from a Git repo or pull down a tarball from another source. See the Source section for more information.
What conda-build puts into a package depends on what you put into the build, host, or run sections. See the Requirements section for more information. Conda-build will use this information to identify dependencies to link to and identify the run requirements for the package. This allows conda-build to understand what is needed to install the package.
Once the content is downloaded, conda-build runs the build step. See the Build section for more information. The build step runs a script. It can be one that you provided. See the Script section for more information.
If you do not define the script section, then you can create a build.sh or a bld.bat file to be run.
When the build environment is created, it is in a placeholder prefix. When the package is all bundled up, the prefix is set to a dummy prefix. When conda is ready to install the package, it rewrites the dummy prefix with the correct one.
Once a package is built, conda-build will test it. To do this, it creates another environment and installs the conda package. The form of this prefix is:
<path to conda>/conda-bld/<package name + string>/_test_env_placeholder…
At this point, conda-build has all of the info from the meta.yaml about what its runtime dependencies are, so those dependencies are installed as well. This generates a test runner script with a reference to the testing meta.yaml that is created. See the Test section for more information. That file is run for testing.
After the package is built and tested, conda-build cleans up the
environments created prior and outputs the metadata. The recipe for
the package is also added in the output metadata. The metadata directory
is on the top level of the tarball in the
The metadata contains information about the dependencies of the
package and a list of where all of the files in the package go when
it is installed. Conda reads that metadata when it needs to install.
conda install causes conda to:
reach out to the repo data containing the dependencies,
guess the right dependencies,
install a list of packages,
unpack the tarball to look at the info,
verify the file based on metadata in the package, and then
go through each file in the package and put it in the right location.