Add GroupV1 + InboxV2#92
Conversation
| [dependencies] | ||
| client = { path = "../../crates/client" } | ||
| # Reference a specific commit so updates to the Core does not break examples | ||
| client = { git = "https://github.com/logos-messaging/libchat", rev = "39bf26756448dd16ddff89a6c0054f79236494aa" } |
There was a problem hiding this comment.
This is causing NIX build to crash.
Chat-CLI requires changes however I'd prefer not to deal with that in this PR.
There was a problem hiding this comment.
Rather than this being a blocker, i'm inclined to revert this and find a solution for Chat-CLI in a different PR.
Naively seems to me that the CLI app should be put into its own repo so:
- It can properly pinned to a dependency versions
- Easily forkable / readable by developers
- Not block future development.
- Maintain its own github issues.
There was a problem hiding this comment.
I see chat-cli as a demo for this repo, and I think it's healthy to keep it buildable and runnable on the same revision as core. A few reasons:
- It's the smallest end-to-end consumer of the public API. If a core change breaks it, that's signal the public surface shifted - exactly the feedback we want at PR time, not weeks later.
- Demos are the first thing new contributors touch. A broken or git-pinned demo is a poor first impression and undermines trust in the rest of the codebase.
There was a problem hiding this comment.
I see chat-cli as a demo for this repo, and I think it's healthy to keep it buildable and runnable on the same revision as core. A few reasons:
@osmaczko I agree. There should be an usable app, and it its important its up to date.
Where I disagree is with how coupled these two things are - and the harm they are causing.
Two specific examples:
- Currently deploying a group chat interface requires me to implement UI logic to support multiple conversation members. Never mind working through an application that I am not familiar with to make it compile.
- Adding a new conversationType requires that a database schema be created, and all storage functions/queries implemented or tests fail.
This is need to be done before the PR has even been reviewed. Its not certain that more updates to these components will be required.
These should be kept up to date, however they can occur after the the core PR has landed. This is especially helpful while major changes are occurring.
Potential Solution
It feels we could allow core changes to be deployed, and avoid negative consequences with process.
-
Integration tests allow surfacing of changes to the public API's, while using a simplified types that do not contain application specific logic, Developers must show that these tests pass to be accepted: https://github.com/logos-messaging/libchat/pull/92/changes#diff-f7e99e5ced818e53f5f63b2a952461475a9b1ac25e99ddb1654ffe8bd41a9f40R79
-
We use a gitflow like approach (main, dev, feature_branch) to track the work. Different developers can handle the different tasks, a requirement for merging to main is that all examples and demos function.
-
Or pin larger components to a git ref, and automatically create github issues to update the items and bring them up to date. We set shelling fences at (1 Week, to ensure) they get updated. CI checks run on intervals to detect anything lagging behind.
Pinning/Branches means that the clients are always functional, work is spread across the team and PRs don't need to be 4000 lines. Leading to more thoughtful review.
There was a problem hiding this comment.
- Currently deploying a group chat interface requires me to implement UI logic to support multiple conversation members. Never mind working through an application that I am not familiar with to make it compile.
This can be done in a follow-up step, right? The crucial part is that it builds and runs. It's fine if it keeps supporting only private convos for now.
- Adding a new conversationType requires that a database schema be created, and all storage functions/queries implemented or tests fail.
Given the new conversation type will be covered by UI logic in a follow-up, can't we just stub the storage interface with todo!()?
- Integration tests allow surfacing of changes to the public API's, while using a simplified types that do not contain application specific logic, Developers must show that these tests pass to be accepted: https://github.com/logos-messaging/libchat/pull/92/changes#diff-f7e99e5ced818e53f5f63b2a952461475a9b1ac25e99ddb1654ffe8bd41a9f40R79
I agree, and I believe we should have full coverage of the public API within integration tests.
- We use a gitflow like approach (main, dev, feature_branch) to track the work. Different developers can handle the different tasks, a requirement for merging to main is that all examples and demos function.
- Or pin larger components to a git ref, and automatically create github issues to update the items and bring them up to date. We set shelling fences at (1 Week, to ensure) they get updated. CI checks run on intervals to detect anything lagging behind.
Good ideas, but I don't like either of them. The Gitflow-like approach means dev is allowed to have a broken demo, and pinning larger components to a git ref of the same repo feels odd as it's hard to navigate and reason about the code (inception). I really see two options:
- Keep the demo buildable and runnable, add stubs if needed, follow up with features to keep PRs reasonably sized.
- Move the demo to a separate repo, as you proposed before.
I'm still opting for 1).
| pub trait GroupConvo<DS: DeliveryService, RS: KeyPackageProvider>: Convo { | ||
| fn add_member(&mut self, members: &[&AccountId]) -> Result<(), ChatError>; | ||
|
|
||
| // This is intended to replace `send_message`. The trait change is that it automatically | ||
| // sends the payload directly. | ||
| fn send_content(&mut self, content: &[u8]) -> Result<(), ChatError>; | ||
| } | ||
|
|
There was a problem hiding this comment.
Group chats require extra functionality, compared to 1:1 Chats.
This trait extends trait:Convo to include group management functions, while also requiring Groupchats to implement all the same functions as the other conversations
| account_id: &AccountId, | ||
| welcome: &MlsMessageOut, | ||
| ) -> Result<(), ChatError> { | ||
| let invite = GroupV1HeavyInvite { |
There was a problem hiding this comment.
The version number in file name and struct name is poping up everywhere, consider following options:
- not introduce version identifier in file name.
- use enums for version variants.
There was a problem hiding this comment.
use enums for version variants.
This is a Protobuf message constructor which represents a specific Invite for a specific ConversationType. I'm not sure how you'd want to use an enum here?
There was a problem hiding this comment.
For protobuf message constructor, It can be namespaced like v1::GroupInvite.
Actually most of my concern comes from the version identifier splits in the file names and many other structs.
There was a problem hiding this comment.
Namespacing in general is not a terrible choice here. Something to consider.
However the natural namespace would be inbox as that is the owner of all invites: that means that handling inbound messages will result in a match statement which is all GroupInvites. separated by artificial namespaces.
Using:
inbox::<ConversationTypeName>::Invite is one approach but I'm not sure its improved.
eg:
inbox::group1::Invite
inbox::group2::Invite
inbox::private2::Invite
Flat names would be easier to reason about, and having many items all named the same seems like adding more confusion than clarity
There was a problem hiding this comment.
Actually most of my concern comes from the version identifier splits in the file names and many other structs.
Seems your issue is with the ConversationType naming policy, not the Implementation.
ref: logos-co/logos-lips#323
| welcome: &MlsMessageOut, | ||
| ) -> Result<(), ChatError> { | ||
| let invite = GroupV1HeavyInvite { | ||
| welcome_bytes: welcome.to_bytes()?, |
There was a problem hiding this comment.
Better call it message or welcome_message.
There was a problem hiding this comment.
I don't follow; You are suggesting to rename the proto_buf field to "message"?
There was a problem hiding this comment.
Yes, no need for type bytes, GroupV1HeavyInvite -> message seems express the intention pretty clear.
There was a problem hiding this comment.
Yes, no need for type bytes
I don't agree here. The protobuf does not accept any arbitrary message. It specifically requires a Mls::Welcome that has been encoded into bytes. Calling it a welcome message would imply that the type is a Welcome Protobuf message - where actually it is encoded bytes.
| }; | ||
|
|
||
| let outbound_msg = AddressedEnvelope { | ||
| delivery_address: ProtocolParams::delivery_address_for_account_id(account_id), |
There was a problem hiding this comment.
delivery_address_for_account_id(account_id) looks a bit long, extract deliver address conversion somewhere else (like in the scope of DeliveryAddress) could make it clear. Same for conversation_id_for_account_id.
There was a problem hiding this comment.
Extract deliver address conversion somewhere else (like in the scope of DeliveryAddress)
DeliveryAddress generation is determined by each protocol. InboxV1, InboxV2, GroupV1 all generate delivery addresses differently.
Moving the definition into the generic container DeliveryAddress would defeat the purpose of having multiple versioned protocols.
delivery_address_for_account_id(account_id) looks a bit long,
I can shorten it by some characters if that will help.
There was a problem hiding this comment.
DeliveryAddress generation is determined by each protocol. InboxV1, InboxV2, GroupV1 all generate delivery addresses differently.
Introduce trait and implement by different conversation should solve it clearly.
There was a problem hiding this comment.
I can't say I follow your logic, perhaps you could explain more.
extract deliver address conversion somewhere else (like in the scope of DeliveryAddress) could make it clear.
The stated goal here is clarity.
Introduce trait and implement by different conversation should solve it clearly.
A trait is an interface that exposes functionality. Its a mechanism to define an interface between components.
In this case the delivery address is never used outside of the environment it was defined.
I don't understand how exposing an internal variable or function outside of its natural scope helps with making things clear?
There was a problem hiding this comment.
I can flatten it, into a non member function and make the name less long. Hopefully that can make things more clear.
| ctx: Rc<RefCell<PqMlsContext>>, | ||
| } | ||
|
|
||
| impl<DS, CS, RS> InboxV2<DS, RS, CS> |
There was a problem hiding this comment.
By looking into the implementation, I feels this is specific for group chat based on MLS, there is no need for a generic name like InboxV2, it could be MlsInbox or similar.
The version number here looks confused with InboxV1, before we have a multi version architecture, we should focus on concrete concept for naming.
There was a problem hiding this comment.
I feels this is specific for group chat based on MLS,
It is not.
InboxV2 is a breaking change to InboxV1, which offers different framing, and account support.
/// An PQ focused Conversation initializer.
/// InboxV2 Incorporates an Account based identity system to support PQ based conversation protocols
/// such as MLS.
There was a problem hiding this comment.
Does InboxV2 also works for PrivateV1 in the current implementation or it's only for MLS?
If it's only MLS, make it clear that it's for MLSInbox, then after introduce adapter for PrivateV1, the generic Inbox can be introduced and delegate to the PrivateV1Inbox and MLSInbox.
There was a problem hiding this comment.
Does InboxV2 also works for PrivateV1
No it does not. PrivateV1 does not use PQE or LogosAccounts.
or it's only for MLS?
No it is not. It would support any any conversationType that is intended to use LogosAccounts.
| fn cleanup_old_skipped_keys(&mut self, max_age_secs: i64) -> Result<usize, StorageError>; | ||
| } | ||
|
|
||
| // TODO: (P2) this should be defined in the ConversationType |
There was a problem hiding this comment.
There are many todo items in the PR, it should be better tracked in the issue with pending tasks, not in the committed code, as there is not enough context to understand the todo item, so it's not helpful for code readers.
There was a problem hiding this comment.
Yes all TODOs must have a corresponding issue.
| use crate::conversation::{GroupV1Convo, IdentityProvider}; | ||
| use crate::types::AccountId; | ||
| use crate::utils::{blake2b_hex, hash_size}; | ||
| pub struct PqMlsContext { |
There was a problem hiding this comment.
We can just flatten the fields inside this context to the above struct which is InboxV2, since it's not used anywhere else.
There was a problem hiding this comment.
I see what you are saying, and don't disagree here.
However this is a ContextObject - Its expected to grow. If DeMLS is integrated and the MLSContext is still relatively simple than I could see flattening it being a smart choice.
There was a problem hiding this comment.
By quickly looking into the context object pattern, it looks useful to provide traced info going though the call stack like request id, or other similar shared state.
In our codebase, there are many context objects, generally I don't see strong requirement for so many context.
There was a problem hiding this comment.
I don't see strong requirement for so many context.
There are currently 2 Contexts and soon to be 1 Context:
- Conversations::Context which was an FFI Context pattern. This object has since changed, and as noted in my PR description need to be removed.
Renaming Context. Context is no longer an accurate name, however that can be discussed later.
This PR is already large/contentious; seems like a bad idea to combine with more changes. If It will help get this merged, i'm happy to add that change set to this PR.
- MlsContext Trait + PqMlsContext impl
Which is a context pattern for passing in a growing set of execution context to Mls style Conversations.
or other similar shared state.
Yes, exactly.
If DeMLS is integrated and the MLSContext is still relatively simple than I could see flattening it being a smart choice.
Is this not sufficient?
| impl ChatError { | ||
| // This is a stopgap until there is a proper error system in place | ||
| pub fn generic(e: impl ToString) -> Self { | ||
| Self::Generic(e.to_string()) |
There was a problem hiding this comment.
I believe the practice here is to use Unknown(err) for such catchall error. The comment can be removed too with unknown error and implement From<String> trait.
There was a problem hiding this comment.
The use of a named constructor allows this function to be passed as a Fn, FnOnce reducing complexity at callsites: e.g. result.map_err(ChatError::generic).
There was a problem hiding this comment.
I believe the practice here is to use Unknown(err) for such catchall error.
Correct.
However there is a semantic difference.
Unknown is a truly unknown type.
Generic is being used as a catchall until someone tackles proper error handling. Currently short on dev power.
All Generics MUST be converted to a real type.
| let inbox = Inbox::new(Rc::clone(&store), Rc::clone(&identity)); | ||
|
|
||
| let pq_inbox = InboxV2::new( | ||
| LogosAccount::new_test(name), |
There was a problem hiding this comment.
Raise it again, test identifiers should not made into product code, even it's temporary. It's a bad smell in code. Consider replace it with a meaningful comment.
There was a problem hiding this comment.
test identifiers should not made into product code
It's a bad smell in code.
I completely agree.
should not made into product code, even it's temporary.
I don't consider this production code at the moment. Specifically the "new_test" is intended to warn others that this code is not to be used, and the function will be removed quickly.
Consider replace it
To resolve this issue is going to make this PR significantly larger, but I'm happy to do that if desired.
There was a problem hiding this comment.
I mean "product" code not production, which means the logic for features, specifically not for test.
Specifically the "new_test" is intended to warn others that this code is not to be used
A clear comment serve this purpose better, not a *test identifier.
| types::{AddressedEncryptedPayload, ContentData}, | ||
| }; | ||
|
|
||
| pub trait IdentityProvider: OpenMlsSigner { |
There was a problem hiding this comment.
Is this going to be shared with others or only group_v1? Since it's re exported in conversations module, need to make it clear about the scope of this trait.
There was a problem hiding this comment.
Is this going to be shared with others or only group_v1?
Not exactly sure what we mean by shared.
IdentityProvider is an interface that will need to be implemented by some other code, so its required to be accessible by them.
Other conversation types in the future will likely use this trait - the conversation that supports DeMLS for example will likely reuse this trait.
need to make it clear about the scope of this trait.
The definition is owned by GroupV1, and accessible by objects with access to conversations internals. Comment added.
There was a problem hiding this comment.
If owned by GroupV1, it should be GroupIdentityProvider, if shared by others, consider move it outside of group_v1.rs, and reduce constraint of OpenMlsSigner as MLS related type is specifically for group chat.
There was a problem hiding this comment.
consider move it outside of group_v1.rs,
This is valid.
I can move it arbitrarily to InboxV2 where it belongs, however I've often been accused of premature optimization 😆 - so I tried to keep it as local and contained as it should be now and move it when there is good reason.
, and reduce constraint of OpenMlsSigner as MLS related type is specifically for group chat.
These two problems have been solved for GroupV2 however backporting here would increase lines by another 500 lines.
Is that what you'd like?
| fn public_key(&self) -> &Ed25519VerifyingKey; | ||
| } | ||
|
|
||
| pub trait MlsContext { |
There was a problem hiding this comment.
Consider flattening this if possible and avoiding the Context naming. Context can be a very overloaded abstraction and sometimes hides what the trait actually represents. A more specific name may make the API clearer.
There was a problem hiding this comment.
Context can be a very overloaded abstraction and sometimes hides what the trait actually represents.
Unfortunately the design pattern being used in this case is the 'Context Object Pattern'. The name specifically refers to that. I can rename it, but it would be less clear about the intention.
Consider flattening
I'm not sure how you would like to flatten a inter-module interface? The trait defines required functionality that needs to be supplied by the caller. I'm not sure how to remove a layer of nesting, when there is only one layer
A more specific name
Its the Context for all Mls Based chats. Alternatives I can see:
GroupV1MlsContext: Reflects that its currently only used in GroupV1MlsContextObject: This seems worse, as its not an object its a trait.
Do either of these align with what you are looking for
There was a problem hiding this comment.
I may not make myself clear.
I mean you are consolidate a lot things in the context trait, it should be split into different ones for the specific feature it provides, for example a identity provider trait could be used, not putting it in a generic context.
So instead have a big trait for a lot of things, there are many specific traits for the feature it provides. Or if you think the features are related and want to consolidate, a better name needed to express the relation / consolidatation, not a generic context, which has been over used in my opinion.
There was a problem hiding this comment.
Based on your feedback I ended up renaming it MlsProvider in my working branch, - Not sure if that helps at all.
So instead have a big trait for a lot of things, there are many specific traits for the feature it provides.
Thats going to be hard. OpenMls accepts a 'Provider' object which has cryptography, storage into one. At somepoint those functionalities are going to need to be implemented together. As mentioned elsewhere Accounts are moved outside of the conversations crate all together which simplifies this object. But the concept still remains
| } | ||
|
|
||
| // Constructs a new conversation upon receiving a MlsWelcome message. | ||
| pub fn new_from_welcome( |
There was a problem hiding this comment.
Is join fit better here?
Similarly create looks better for new.
There was a problem hiding this comment.
Not sure I agree: 'new_' is the convention for constructing new instances of a type. Join would imply an operation on a GroupV1Convo rather than creating a new instance.
| let Some(mls_group) = MlsGroup::load(ctx.borrow().provider().storage(), &group_id) | ||
| .map_err(ChatError::generic)? | ||
| else { | ||
| return Err(ChatError::NoConvo("mls group not found".into())); |
There was a problem hiding this comment.
ok_or_else on option can inline the logic with functional style.
There was a problem hiding this comment.
updated to:
let mls_group = MlsGroup::load(ctx.borrow().provider().storage(), &group_id)
.map_err(ChatError::generic)?
.ok_or_else(|| ChatError::NoConvo("mls group not found".into()))?;
osmaczko
left a comment
There was a problem hiding this comment.
Cute. I see it as a good step forward.
| @@ -27,13 +27,15 @@ pub trait EphemeralKeyStore { | |||
| pub enum ConversationKind { | |||
| PrivateV1, | |||
| Unknown(String), | |||
There was a problem hiding this comment.
Can we keep Unknown either first or last?
| /// on registration; others fetch it to initiate a conversation. | ||
| pub trait RegistrationService: Debug { | ||
| type Error: Display; | ||
| fn register(&mut self, identity: &str, key_bundle: Vec<u8>) -> Result<(), Self::Error>; |
There was a problem hiding this comment.
| fn register(&mut self, identity: &str, key_bundle: Vec<u8>) -> Result<(), Self::Error>; | |
| fn register(&mut self, identity: &AccountId, key_bundle: Vec<u8>) -> Result<(), Self::Error>; |
There was a problem hiding this comment.
Good eyes; It should definitely be AccountId
| macro_rules! hash_sizes { | ||
| ($($(#[$attr:meta])* $name:ident => $size:ty),* $(,)?) => { | ||
| $( | ||
| $(#[$attr])* | ||
| pub struct $name; | ||
| impl HashLen for $name { type Size = $size; } | ||
| )* | ||
| }; | ||
| } |
| pub trait GroupConvo<DS: DeliveryService, RS: KeyPackageProvider>: Convo { | ||
| fn add_member(&mut self, members: &[&AccountId]) -> Result<(), ChatError>; | ||
|
|
||
| // This is intended to replace `send_message`. The trait change is that it automatically |
There was a problem hiding this comment.
| // This is intended to replace `send_message`. The trait change is that it automatically | |
| // TODO: This is intended to replace `send_message`. The trait change is that it automatically |
| } | ||
|
|
||
| #[test] | ||
| fn create_group() { |
There was a problem hiding this comment.
Would be great to add assertions to this test.
| } | ||
|
|
||
| #[derive(Clone, Debug)] | ||
| pub struct LocalBroadcaster { |
There was a problem hiding this comment.
Should InProcessDelivery be removed then?
There was a problem hiding this comment.
Eventually that would make sense. Or InProcess could be improved better than LocalBcaster.
I could't use InProcess because it didn't support subscriptions, which were needed.
Currently the Client requires a functional change to use Subscriptions if its going to update to LocalBcaster.
So I chose to leave it as is, and then update it when possible.
| /// needs to build a [`CredentialWithKey`]: `friendly_name` becomes the | ||
| /// `BasicCredential` label and `public_key` becomes the signature-verification key. | ||
| pub trait IdentityProvider: OpenMlsSigner { | ||
| fn friendly_name(&self) -> String; |
There was a problem hiding this comment.
| fn friendly_name(&self) -> String; | |
| fn display_name(&self) -> String; |
| [dependencies] | ||
| client = { path = "../../crates/client" } | ||
| # Reference a specific commit so updates to the Core does not break examples | ||
| client = { git = "https://github.com/logos-messaging/libchat", rev = "39bf26756448dd16ddff89a6c0054f79236494aa" } |
There was a problem hiding this comment.
I see chat-cli as a demo for this repo, and I think it's healthy to keep it buildable and runnable on the same revision as core. A few reasons:
- It's the smallest end-to-end consumer of the public API. If a core change breaks it, that's signal the public surface shifted - exactly the feedback we want at PR time, not weeks later.
- Demos are the first thing new contributors touch. A broken or git-pinned demo is a poor first impression and undermines trust in the rest of the codebase.
| @@ -0,0 +1 @@ | |||
|
|
|||
There was a problem hiding this comment.
It seems not used anymore.

This PR Introduces MLS Groupchat via GroupV1Convo
Changes:
Remove "bin/chat-cli" from default members.
Having to update Applications when pushing changes for Core Protocol slows down development. Recommend moving to a separate Repo, or lock which can lock to a specific Version.
Moves DeliveryService trait from Client into Core::Context. Adds RegistrationService as well.
Adds InboxV2 which is PQresistant.
Adds GroupConvo Trait which extends Convo with Group Management Operation.
Adds GroupV1Convo which is Service aware - They can interact directly with the DeliveryService and RegistrationService impls. This streamlines the return types from Conversation functions.
Adds
fn load_group_convoandfn create_group_convowhich returnBox<dyn GroupConvo>. This allows clients to directly invoke methods and avoid relaying function calls through ContextWork Deferred: