# Use of semaphores

**URL:** <https://forum.canton.network/t/use-of-semaphores/1172>\
**Category:** App Development\
**Created:** [September 9, 2020, 2:33am UTC](https://forum.canton.network/t/use-of-semaphores/1172 "2020-09-09T02:33:19Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![matt](https://yyz2.discourse-cdn.com/flex034/user_avatar/forum.canton.network/matt/32/589_2.png) [@matt](https://forum.canton.network/u/matt)\
**Post date:** [September 9, 2020, 2:33am UTC](https://forum.canton.network/t/use-of-semaphores/1172/1 "2020-09-09T02:33:19Z")

</div>

I understand what semaphores are for in the traditional sense, but within the context of DAML I am trying to understand the why and mechanics behind them.

Refer to danban

> <https://github.com/digital-asset/danban/blob/4f9e7b8ba715f652b40cc7ea8cab4800b9aef58c/backend/V3_2/daml/Danban/V3_2/Role.daml#L32>

It appears like when I perform a `fetchByKey` on any contract, its the equivalent of applying a global lock on it then releasing the lock after the transaction completes?

How should I be thinking about this?

If I have more than 1 contract to update, then I should always be doing this locking semaphore pattern?

My confusion lies in whats the difference between an `Update` being an all or nothing atomic operation vs also doing this semaphore thing at the start of every choice which is just another way to lock everything to guarantee atomicity?

---

<div class="post-metadata">

**Author:** ![cocreature](https://yyz2.discourse-cdn.com/flex034/user_avatar/forum.canton.network/cocreature/32/36_2.png) [@cocreature](https://forum.canton.network/u/cocreature)\
**Post date:** [September 9, 2020, 6:42am UTC](https://forum.canton.network/t/use-of-semaphores/1172/2 "2020-09-09T06:42:01Z")

</div>

Disclaimer: I’m not too familiar with the Danban model so I might be missing some details.

Generally, this kind of pattern where you guard a choice by the existence of a key (if it doesn’t exist the `fetchByKey` will fail immediately and the transaction will be aborted) can be used to reduce contention. To make things a bit more concrete, in Danban this works as follows:

1. The [`Directory_AddObs`](https://github.com/digital-asset/danban/blob/4f9e7b8ba715f652b40cc7ea8cab4800b9aef58c/backend/V3_2/daml/Danban/V3_2/User.daml#L19) choice modifies all user role contracts associated with the respective keys. This means that it is very likely to run into contention issues if any of the contracts are modified in parallel (if you have lots of users, this also becomes a large transaction and thereby necessarily slower which makes it even more problematic).
2. To avoid this, there is a [`PauseApp`](https://github.com/digital-asset/danban/blob/4f9e7b8ba715f652b40cc7ea8cab4800b9aef58c/backend/V3_2/daml/Danban/V3_2.daml#L26) choice. This will archive the semaphore contract causing all the choices you discovered which could cause contention to fail immediately.
3. This is then used as follows:
  1. Pause the app
  2. Run the Transaction likely to run into contention issues which will now succeed since everything else fails immediately.
  3. Unpause the app.  
You can find an example of this at thttps://github.com/digital-asset/danban/blob/0064d0e41506baa1b86a6f24fb6ecc26c82892df/src/server/ledger.js#L169

You might notice that I link to an older model here. This is because as far as I know it is crucial to do the first step in a separate transaction. Otherwise, if pausing the app is part of the same transaction as the problematic choice you run into the exact same issue as the other transactions will only fail once the problematic one has been committed. So I don’t think the [`UserSessionAck`](https://github.com/digital-asset/danban/blob/4f9e7b8ba715f652b40cc7ea8cab4800b9aef58c/backend/V3_2/daml/Danban/V3_2.daml#L62) choice in the current model is actually useful.

As far as I can tell, `PauseApp` is only ever used in the same transaction in the current model which makes this pattern useless. But I could definitely be missing something. Maybe @bernhard and @DFeichtinger can shed some light on this.

---

<div class="post-metadata">

**Author:** ![matt](https://yyz2.discourse-cdn.com/flex034/user_avatar/forum.canton.network/matt/32/589_2.png) [@matt](https://forum.canton.network/u/matt)\
**Post date:** [September 9, 2020, 7:58am UTC](https://forum.canton.network/t/use-of-semaphores/1172/3 "2020-09-09T07:58:50Z")

</div>

Thanks for the insights,

The reason I ask about this semaphore pattern is id like to assign parties to role groups. There has been a few threads on this

> [@User Access, Authentication , Access Token and DAML Roles -](https://discuss.daml.com/t/user-access-authentication-access-token-and-daml-roles/853/2):
>
> It looks like you have a pretty good handle on your problem, and either of those two will work. There is a third option you can include in your considerations, and that is to represent each party’s role independently on the ledger. So rather than having a single DevRole contract and a single ManagerRole contract each with a set of parties, instead you have DevRole and ManagementRole templates and create a single instance per party when/while each party is assigned that role. The downside of this…

For example, my app is a phone book with 1000+ pages.

Lets assume its hosted on DABL so if it takes off everyone is their own unique party. I don’t know of a nice way to group N parties to 1 group type role without experiencing this phenomena I am coining `the observer propagation quandary` TM 😛

A newly onboarded party is a basic user by default, I need to give them the ability to fetch every phone book page so my goal is to make them a stakeholder of all 1000+ phone book pages.

To achieve this, I need to do a bulk update and propagate this newly onboarded user to the observer list of all 1000+ phone book pages. (This is the part I am concerned over and if there needs to be some form of global lock in place).

I have other specific roles but these arent an issue because I have single role contracts for individual parties so they can do role membership checks by simple `lookupByKey` which I really like.

Is there a solution to this quandary or something to make me feel at ease that I am not doing something really dodgy?

---

<div class="post-metadata">

**Author:** ![DFeichtinger](https://yyz2.discourse-cdn.com/flex034/user_avatar/forum.canton.network/dfeichtinger/32/127_2.png) [@DFeichtinger](https://forum.canton.network/u/DFeichtinger)\
**Post date:** [September 9, 2020, 8:58am UTC](https://forum.canton.network/t/use-of-semaphores/1172/4 "2020-09-09T08:58:04Z")

</div>

My understanding is broadly the same, but just a minor point: when adding users they need to be added as observers to the semaphore. Unpausing after adding the user fetches the updated user list 🙂

---

<div class="post-metadata">

**Author:** ![bernhard](https://yyz2.discourse-cdn.com/flex034/user_avatar/forum.canton.network/bernhard/32/356_2.png) [@bernhard](https://forum.canton.network/u/bernhard)\
**Post date:** [September 10, 2020, 7:35am UTC](https://forum.canton.network/t/use-of-semaphores/1172/5 "2020-09-10T07:35:27Z")

</div>

@cocreature is spot on regarding the original purpose of the `Semaphore` contract. @Andreas_Lochbihler has just written an in-depth piece of documentation on DAML’s causality model. It explains the ordering guarantees a DAML Ledger gives, and consequently the types of contention you can run into.

Generally speaking, creates and archives are high contention, non-consuming actions are not. If you have 100 transactions all fetching or looking up a contract, and one archiving it, it doesn’t matter in which order the 101 transactions are received, the archive will always go through. It’s the fetches and lookups that come after which will fail.

Contention is controlled in Danban by sharding almost all data and using nonconsuming exercises where possible. The only times you have contention is if the same object is modified by two transactions - eg two updates to the same card.

The exception is the addition of new users. Because of the “observer propagation quandary”, a transaction adding a new user has contention with a lot of other operations. To have a chance of committing it, the operator has the `Semaphore` to pause the app. Because the `lookupByKey` is non-consuming, the archival/create of the `Semaphore` gets priority.

Right now, there is no good way to solve the “observer propagation quandary”, but it’s an issue high up in our priorities of problems to solve. The first solution we are planning is a feature which allows party-sharing. For your use-case, you’d create a `directory` or `public` party, and then give all users read-access to that party. When they query the Ledger, they’ll then get data for both their own as well as the `directory` party, and when they submit commands, the contracts known to `directory` can be fetched and looked up.

This mechanism relies on the `directory` party being hosted at least read-only on the same node, which makes it slightly more complicated in fully decentralized topologies, but we will also make it possible for a party to be hosted on multiple nodes, so you’d just give all relevant nodes read-access hosting for the party.

---

<div class="post-metadata">

**Author:** ![matt](https://yyz2.discourse-cdn.com/flex034/user_avatar/forum.canton.network/matt/32/589_2.png) [@matt](https://forum.canton.network/u/matt)\
**Post date:** [September 10, 2020, 8:16am UTC](https://forum.canton.network/t/use-of-semaphores/1172/6 "2020-09-10T08:16:15Z")

</div>

Thanks for the reply, I really look forward to a party sharing solution.

> [@bernhard](#):
>
> For your use-case, you’d create a `directory` or `public` party, and then give all users read-access to that party. When they query the Ledger, they’ll then get data for both their own as well as the `directory` party, and when they submit commands, the contracts known to `directory` can be fetched and looked up

Is this achievable on DABL?

Just trying to establish a mental model for all the moving parts to make it work.

1. create a `BasicUser` party in DABL and lets say their unique party id is `ledger-party-basic-user’

This `BasicUser` is then a static observer added on any contract that has the ability to be available to basic users solving the observer propagation quandary.

1. if alice then uses my DABL app, she will get a token issued to her by DABL looking like this.

My question is, I need to append the `BasicUser` party to the `actAs` list for it to have a chance to work? There doesn’t seem to be a way in DABL to change it directly (yet)?

```haskell
{
  "iss": "projectdabl.com/login",
  "sub": "blah",
  "exp": 1599811181,
  "https://daml.com/ledger-api": {
    "actAs": [
      "ledger-party-alice"
    ],
    "applicationId": "DABL",
    "ledgerId": "blah"
  },
  "partyName": "alice",
  "rights": [
    "read",
    "write:create",
    "write:exercise"
  ]
}

```

---

<div class="post-metadata">

**Author:** ![bernhard](https://yyz2.discourse-cdn.com/flex034/user_avatar/forum.canton.network/bernhard/32/356_2.png) [@bernhard](https://forum.canton.network/u/bernhard)\
**Post date:** [September 10, 2020, 9:28am UTC](https://forum.canton.network/t/use-of-semaphores/1172/7 "2020-09-10T09:28:14Z")

</div>

The feature is WIP and there’s still design work to be done. The only thing I’m sure of is that whatever we come up will work on DABL. It’s one of the main use cases Maybe @dtanabe or @dliakakos already have some idea what direction it’ll go.

---

<div class="post-metadata">

**Author:** ![dtanabe](https://yyz2.discourse-cdn.com/flex034/user_avatar/forum.canton.network/dtanabe/32/91_2.png) [@dtanabe](https://forum.canton.network/u/dtanabe)\
**Post date:** [September 17, 2020, 2:02am UTC](https://forum.canton.network/t/use-of-semaphores/1172/8 "2020-09-17T02:02:17Z")

</div>

On DABL, every ledger gets a “public party”; all users can freely acquire a read-only token for that `Party` and can then consequently read any data with that party as an observer.

This flow is documented here: [https://docs.projectdabl.com/api/#public-party](https://docs.projectdabl.com/api/#public-party)