I have been thinking a bit about how to tackle the lack of support for external modules within the Salesforce packaging toolset. By external modules I really mean external to the source tracking repo that you are working on. I have partially implemented an approach in some of my own tooling but thought it would be worth posting about so others can chip in and improve.
Introduction
External module dependencies are declared in an extension to the Salesforce sfdx-project.json schema. As an extension it uses the “plugins” property which is open for use by third-parties to add additional configuration data to sfdx-project.json.
An external dependency is metadata that is available for use but not described by the “packageDirectories” entries of sfdx-project.json. The dependency metadata can only be provided as source although that may be in either MDAPI or SFDX format (aka source format). The location of the metadata may be described in multiple ways, via filesystem path, via url or via npm module name.
The purpose of declaring these project dependencies is to allow developer & CI tooling to have full visibility of the metadata that may need to be deployed to an org, but crucially to maintain the flexibility to compose that metadata from a number of separately managed modules.
There is some overlap in objectives here with Salesforce’s own second generation packaging model. The distinction is that external module dependencies do not imply any particular form of packaging is in use. You can choose to package an external modules dependency on its own or to bundle it with other modules and application code into a package in either first or second generation form.
Extension Mechanism
In sfdx-project.json we include a “dependencies” array within the “plugins” property to indicate modules that can be imported.
“plugins”: {
“dependencies”: [
{namespace: “xyz”, “path”: “modules/fflib-apex-common” target=”force-app/common”}
]
}
All dependencies must identify a source for the metadata, either as a “path” or “npm” property, and a target path. The source and correct version for a “npm” metadata is assumed to have been pre-loaded in the node_modules directory. It is provided as an alternative to “path” to aid version conflict resolution.
An optional namespace may also be provided which may differ from the declared namespace in the sfdx-project.json file of the dependency.
Dependent Module Installation
Dependencies are installed into the metadata via a namespace transformation and file copying into the “target” location. The target must be an existing “packageDirectories/path” directory or descendant path into which the files can be copied without overwriting existing metadata.
As modules may themselves declare dependencies this process must be recursively executed. Where multiple different versions of a “npm” module are identified only the version with the highest semantic version should be installed.
Namespace transformation is applied where the “namespace” property is provided and different from the namespace property in the modules sfdx-project.json file if it has one. The transform changes all occurrences of ‘<namespace>__’ & ‘<namespace>.’ from the declared namespace to the target namespace.
Module installation may create name clashes within the metadata, such as two Apex classes of the same name in different folders under the same “packageDirectories/path” directory. While these may be detected during an install they can also be ignored as package creation will identify them.
Dependent Module Cleaning
Dependent modules are cleaned from the “packageDirectories/path” directories by following the same approach as ‘Installing’ but replacing the file copying operations by deletions of the target file.
@nawforce 