SCU: a command can run under a package version the client was not compiled against

App Development
, ,
1

One observation from running Smart Contract Upgrade on a sandbox, with the exact errors, in case it saves someone a debugging session.

Setup: dpm 1.0.22, Daml SDK 3.4.11, single-participant dpm sandbox (its log banner says Starting Canton version 3.5.18), Daml Script 3.4.11 and the JSON API. Two versions of one package, 0.1.0 and 0.2.0, the second adding a trailing Optional field to one template; dpm upgrade-check --both passes for the pair.

The rule itself is documented: with no package_id_selection_preference, the participant resolves the package name to the highest vetted version. What the documentation does not show is how that looks from the client.

After 0.2.0 was vetted, a Daml Script compiled against 0.1.0 submitted an exercise with no preference. The ledger executed the 0.2.0 code and the resulting contract carried the 0.2.0 package id. The only failure was on the client, when the script runner could not translate the result: Failed to translate create argument: Lookup(NotFound(Package(00a85a8b...))). The ledger side effect happened under a version the client had never seen, and the client’s error says nothing about versions.

The mirror case: a script compiled against 0.2.0 and run before 0.2.0 was uploaded was not rejected. The participant resolved the name to the only vetted version, 0.1.0, executed that, and again only the runner failed, with Lookup(NotFound(Package(3b3ac851...))). So “package not vetted” never appears as an error in this configuration.

Pinning works as documented: packagePreference in Daml Script, packageIdSelectionPreference on the JSON API. All of this is on one participant; we have not looked at how it behaves across participants.

One question, mostly for the Digital Asset folks here: is there a recommended way for a client to state the version it was compiled against, so that a mismatch fails at submission rather than executing under another version and failing only on result translation?

2

Not from DA so they’ll still confirm, but yes.

In Daml Script, that’s exerciseExactCmd and the other *ExactCmd variants. The plain commands strip the package id and send a by-name template id, which is why the participant gets to pick.

The exact variants keep the id of the package the script was compiled against, so there is nothing to resolve: it runs under that version, and if the participant doesn’t have it you get NOT_FOUND: PACKAGE_NOT_FOUND at submission instead of the translation error.

Same effect as your packagePreference pin, but the id comes from the DAR instead of being passed in.

3

@crisog thanks, that was the missing piece. We ran it on a fresh sandbox (Canton 3.4.11 this time, Daml Script 3.4.11, a package whose 0.2.0 adds one trailing Optional field):

  • A 0.2.0 script using createExactCmd against a ledger with only 0.1.0 vetted failed at submission: NOT_FOUND: PACKAGE_NOT_FOUND ... Couldn't find package cc1d3bc4... while looking for template or interface cc1d3bc4...:Demo:Counter.
  • A 0.1.0 script using exerciseExactCmd after 0.2.0 was vetted ran under 0.1.0, and the new contract kept the 0.1.0 package id.
  • On the JSON API the pin comes from the template id: <package-id>:Demo:Counter ran under 0.1.0, #scu-exact-demo:Demo:Counter under 0.2.0.

Two things we did not expect. A plain createCmd from the 0.2.0 script, with the new field left empty and only 0.1.0 vetted, failed nowhere: the contract was created under 0.1.0 and the script reported success. So a create is quieter than the exercise case in the first post. And the pin has a cost: exerciseExactCmd from the 0.1.0 script on a contract created under 0.2.0 with the new field set failed with INTERPRETATION_UPGRADE_ERROR_TRANSLATION_FAILED ... to a value of type Demo:Counter@e703b7d1 fails. Pinning turns a silent version switch into a hard failure once the ledger holds contracts the pinned version cannot read.

For the DA folks: for a production client that must not run under a version it was not built against, is the package-id template id the intended tool, or packageIdSelectionPreference?

4

Hi @woof-software,

Re: For a production client that must not run under a version it was not built against, is the package-id template id the intended tool, or packageIdSelectionPreference?

Given your scenario where you are trying to exercise commands for both 0.1.0 and 0.2.0 packages, in which both are required for different usages, package-id template-id would be the recommended workflow.

As your intended workflow is via Daml Script, where you have more control over exactly what you are trying to do, it would be better to use package-id template-id in this scenario, to explicitly state the package-id template-id you want to use, and to ensure it never refers to the wrong package ID.

As for usage of, “packageIdSelectionPreference”, this would be used during workflows which rely on package-name references. In the sense that all submitted commands which specify only the package-name, will then use the package-id as specified from this field.

Some further documentation for reference:

Best Regards,

Solomon

5

Thanks, @solomonchauda, that settles which tool to use: an explicit package-id template id where a client must run under the version it was built against, and packageIdSelectionPreference for workflows that submit by package name.

One follow-up on the cost we saw: a client pinned to 0.1.0 fails with INTERPRETATION_UPGRADE_ERROR_TRANSLATION_FAILED once the ledger holds contracts created under 0.2.0 with the new field set. Is the intended practice to roll pinned clients forward before anything starts creating 0.2.0 contracts, or is there another pattern you would recommend?

6

Hi @woof-software please see Smart Contract Upgrade (SCU).

“When extending data in a Daml model, SCU requires the old model to be representable in the new model. For extending a record, we can only add nullable (Optional) fields, so that old data can be represented by setting these fields to None.”

This means that you would need to add the missing nullable optional fields to the old contracts. Please try this then advise back.

FYI error details: Error Code Reference - Canton Network Docs

Regards,

Mitesh

7

@mitesh.mistry, in our case 0.2.0 already adds the field as a trailing Optional and upgrade-check passes for the pair, so old contracts are read under 0.2.0 without problems.

The failure we described is the other direction: a client pinned to 0.1.0 exercising a contract created under 0.2.0 with that field set to Some, which the engine rejects as a downgrade. For our deployment notes we take this as the rule: clients pinned to the older version move to 0.2.0 before anything populates the new field. Thanks to both of you for the pointers.