Approach to ConversationListBox

Goal of this document is to describe the steps involved in moving from a Textview based widget to a ListBox based widget and to describe to the project what the plan behind the coming PRs and commits will be, also requested by @lovetox.

The document is stored in wiki form to allow discussion and mutual manipulation of the document.

For reference, here's the relevevant discussion from the Gajim MUC (where erik == @ehuelsmann):

[11:22:39 PM] lovetox: erik, yes refactoring some things before working on the listbox is a good idea
[11:23:14 PM] lovetox: its best you write down where you see potential for refactoring and let me look
              over it
[11:23:58 PM] lovetox: so i can tell you into how much of a rabbit hole you are getting your self :)
[11:24:48 PM] erik: lovetox: thanks. I've started some initial explorations in that area. Are you asking
              that I wrote something like a refactoring plan before going into the actual effort?
[11:24:49 PM] lovetox: and of course the smaller the PR the faster it gets merged
[11:24:56 PM] erik: I think that's fair
[11:25:38 PM] erik: Ok. I'll write a bit of a plan and work on small commits along the plan.
[11:26:04 PM] lovetox: yeah i think its usefull if you tell me what and where you want to refactor before
              you start it
[11:33:25 PM] erik: I'll need a few days to get more of a grip on the code base. 

High level assessment of the existing code base

Functional overview

ConversationTextview is tightly coupled with HtmlTextview as it highly depends on the fact that the HtmlTextview is an editor: it directly accesses the editor buffer and uses it all over the place.

On the other side, HistoryWindow and ChatControlBase depend on ConversationTextview to display the list of messages, where HistoryWindow uses some of the api provided by ConversationTextview to populate the window and does some formatting of its own. ChatControlBase leaves formatting of messages entirely to ConversationTextview.

The view performs the following functions:

• Maintaining a mapping of message IDs to text buffer positions
• Maintaining a mapping of 'time' to text buffer positions
• Maintaining a list of corrected messages (by message ID)
• Formatting message lines:
  • Adding encryption status
  • Adding a timestamp
  • Adding the nick of the sender
  • Scanning and formatting URLs in message text
• Formatting non-message lines (such as status messages, muc title, etc.)
• Deal with all user-interactions through the registration of various event handlers (popup menu, clicking of URLs, etc.)

The HtmlTextview adds a single function to the above: parsing the provided content for xhtml-im content (and cleaning non-compliant content).

In addition to ConversationTextview and HtmlTextview, GroupChatControl also influences rendering of messages by selectively passing into the ConversationTextview the fact that a message has an XHTML-IM presentation.

Technical overview

ConversationTextview maintains the following API to achieve the formatting functions:

• print_empty_line
• print_conversation_line
• print_encryption_status
• print_real_text
• print_time
• print_name
• print_subject
• get_end_mark
• get_insert_mark
• (creation and updates of HtmlTextview.get_buffer()-tags in various places)

For the mappings it maintains the following functions and attributes:

• correct_message [function]
• message_list [attribute]
• last_received_message_id [attribute]

HtmlTextview manages presentation with the following functions:

• update_tags
• display_html (which is also responsible for handling xhtml-im)

Concluding

From the summary above, it seems like ConversationTextview is all of the model, the view and the controller: it maintains the data to be presented in all its aspects (model), it generates/manages presentation by virtue of the various print_* methods (view) and it deals with various signals (e.g. user input) which affect presentation as well as other actions such as starting external browsers and all.

This mixture of roles of the ConversationTextview complicates replacing the presentation of messages with a ListBox from a Textview.

Approach to moving to a listbox

1. Separate the concerns currently concentrated in ConversationTextview a. Add new setup to this document a. Agree on changed separation of concerns with @lovetox (and others?)
2. Gradually implement the new separation
3. Based on separated concerns, design a ConversationListBoxview and ListBoxrow
4. Implement minimal Listboxrow and ConversationListBoxview
5. Synchronize APIs between ConversationListBoxview and ConversationTexview
6. Based on some developer setting make listboxview and textview selectable
7. Work on primary ListBoxrow until it's on-par with the current textview approach
8. (merge the on-par replacement to master?)
9. Develop specialized ListBoxrows for status messages, subject lines, etc., etc.

Requirements for a new design

High level functional requirements

• Allow for progressive introduction of improved presentation of various types of "messages" (rows)
• Allow swapping in/out of the current Textview based on developer need (and allow experimentation with other presentations in the future)
• Be on-par presentation-wise with the existing view (before being merged to master)

Technical design

• Separate the concerns (at least) to be able to swap various presentation experiments quickly and easily
  • presentation
  • managing the presented content & reacting to (user) input
• Building presentation logic which maps various types of messages to their respective ListBoxrow classes to allow differences per message(type)

Concluding

(to be done)

Addendum

In a chat with @lovetox, the following requirements were mentioned as well:

[04:46:27 PM] lovetox: first goals should be, the listbox should provide a way to add a message row,
              the message row itself should be a object that displays an avatar, the message,
              encryption symbol
[04:47:25 PM] lovetox: the message row object should be based on some generic object, we will probably
              need many row objects that slightly differ
[04:47:43 PM] lovetox: like some kind of status messages, error messages, info messages, etc
[04:49:23 PM] wurstsalat: File transfers, image previews :)

Design of the end state

Functional requirements

In the end state, it should be easy to experiment with various presentations, where it's possible to swap presentations for each type of message ('status', 'error', 'info', 'incoming', 'outgoing', ...) separately. This will allow better future development and adaptation to user expectations.

Also, the content model should provide the view with chat status data: participants' read receipts (last message read) and typing-or-not status updates.

Technical requirements

• Presentation of both messages from the server stream as well as (informational) messages originating from the client
• Separation of management of content of the chat window from rendering of it
• Separation of presentation and chat management
• Efficient updating of chat content driven by the chat content management class
• The class responsible for managing the content should be able to look up corrected content by the stanza_id of the original stanza (even over the course of multiple updates)
• Presentation layer will be provided all required input to determine correct rendering; currently this amounts to:
  • delayed
    The server indicated that the message delivery was delayed for one reason or another (sending history or other reasons)
  • kind
    The type of conversation event; one of 'error', 'warn', 'info', 'out'/'outgoing', 'in'/'incoming', 'status', 'incoming_queue',
  • xhtml
    The xhtml text in a message in case there is any
  • correct_id
    The msg_stanza_id of the message for which this message is a correction
  • encrypted
    When True, indicates that the message was encrypted on receipt
  • encryption_details
    Contains keys which explain which encryption was used, finger print, (more?)
  • text
    The plain text in a message in case there is any
  • time
    Time the message was received/generated, used for sorting the messages in the view
  • contact
    The sender of the message
  • displaymarking
    Support for markings as specified by XEP-0258
  • subject
    When set, includes a line containing "Subject: ..." followed by an empty line in the conversation view
  • old_kind
    Used to determine if two consecutive rows may be joined as lines from the same nick (under the 'chat_merge_consecutive_nickname' setting) -- decision should be moved to the ContentModel; old_kind is available to the model and can be dropped
  • other_tags_for_name -- likely needs to move to the view/renderer
  • other_tags_for_time -- likely needs to move to the view/renderer
  • other_tags_for_text -- likely needs to move to the view/renderer Suppresses adding the time in front of the chat line
  • graphics
    Indicates that emojis are to be rendered graphically; other than that just passed on to the extension point; should be moved to the view(?)
  • msg_log_id
    Not passed to the view -- equals the row number in the database where this message is saved
  • msg_stanza_id
    The stanza_id as generated by the source to identify the message
  • count_as_new
    When false, indicates that a message should not generate attention even if it fits the requirements, because the message isn't to be considered "newly received"
  • restored_message [derived value]
    This value indicates whether a message comes from before the chat was opened (restored from history or retrieved from MAM)

Required roles/classes

• GroupChatControl/PrivateChatControl/ChatControl
  These classes determine which messages are applicable to the current chat; they do nothing more than add the message to the ConversationContentModel.
• ConversationContentModel
  This class is responsible of maintaining the exact content to be shown in the correct order. This means that any messages being corrected, get updated in this content representation. Additionally, the decision to merge two stanzas into one chat line (because they originate from the same jid and the user enabled merging) is on this class. It's the responsibility of this model indicate to the view it has been updated/appended, including which messages were updated, by stanza_id. It is the responsibility of this class to store and update history of the channel (as this class is the place where it all gets together). Every line contained in the model has a unique ID to find it. In case a line doesn't have a stanza_id (e.g. because it's a status message or because it's a merged line), this class will generate an id and return that to the caller.
• ConversationContentLine
  This class-or-dict contains the available fields from the list above (can it just be a dict or do we need a class?)
• ConversationTextview (current presentation)
  This class is responsible for showing any content available in the model and updating the display when the content model gets updated. It's up to the view to determine how any given message is to be rendered, meaning that the content model should be able to provide all data that might influence this (see the listed fields above for the current situation). The view maintains a mapping between stanza_ids and regions in the widget to ensure efficient updates. Note that this class and the next don't differ in their responsibilities.
• ConversationListBoxview (desired presentation)
  • ConversationStatusListBoxrow
  • ConversationMessageListBoxrow

Note that this design will allow - when the ContentModel will be connected to a source storing history - browsing back in history, if the content model gets extended to read messages from history upon request.

Examples flow of events

Assuming John has joined the John-and-Jane MUC some time ago and he's chatting with Jane (as in: we are well past the initialization/startup phase or "joining process"). Jane responds to John's last message. When John's client receives the message, the following happens:

1. Gajim parses the stream event
2. The app.events system calls the event handlers for muc-gc-message-received
3. The various instances of MUCs in John's client receive the event
4. The instance of the John-and-Jane MUC recognizes the message as applicable
5. The John-and-Jane MUC instance calls its ConversationContentModel's add_message() function
6. The ConvContentModel recognizes this message as being new and adds the function to its history
7. Then the model calls the ConversationView's (either a ConversationTextview or ConversationListBoxview) add_message() function with the new message
8. The view renders the message provided at the appropriate position (in this case, the bottom of the message list)

After a bit of chatting, Jane realizes that she made a typo in the message that John received in the last example. She decides to correct it and send it to John. When John's client receives the message, the following happens:

1. Gajim parses the stream event
2. The app.events system calls the event handlers for muc-gc-message-received
3. The various instances of MUCs in John's client receive the event
4. The instance of the John-and-Jane MUC recognizes the message as applicable
5. The John-and-Jane MUC instance calls its ConversationContentModel's add_message() function
6. The ConvContentModel recognizes this message as existing and being corrected
7. The model updates the list of messages with the new message in place of the old one (retaining a copy of teh old one "inside" the new one)
8. Then the model calls the ConversationView's (either a ConversationTextview or ConversationListBoxview) update_message() function with the old message's id and the corrected message
9. The view renders the message provided at the position where the prior message was rendered, replacing the prior message

Below is work in progress

Design for cleanup of the current state

Responsibilities of roles

The role of the presentation is to:

• Render chat lines coming in from the server
• Render chat lines going out to the server
• Render status updates from the client, server and chat channel (join/leave/etc)
• Update rendered lines when instructed to do so (corrected messages)

which means that the presentation will also need to:

• map message IDs to buffer regions
  • get_end_mark
  • get_insert_mark

The role of the model (ConversationContentModel) is to:

• add_message [function], to maintain:
  • message_list [attribute]
  • last_received_message_id [attribute]
• correct_message [function]

The role of the controller (also ConversationTextview -- for now, but might need to be GroupChat/PrivateChat/ChatControl instead?) is to:

• respond to user interactions
• (open question: to network interactions as well? alternative: the model gets fed directly from the protocol stream)

Impact on the existing code base

The various print_* functions will to be moved from ConversationTextview to HtmlTextview. This impacts:

• gajim/chat_control:ChatControl.restore_conversation()
• gajim/chat_control_base:ChatControlBase.print_conversation_line()
• (to be continued)