Upgrade news data model to note Part 1

[azayati]

Rationale

eXo is working on bringing it's full-featured news publishing system to meeds by leveraging Notes as content storage.

Changes need to be applied to notes prior bringing the News features.

1. Functional requirements

• A News can be created even if there is no Notes app in a space
• A News can be created even if there is the Notes app in a space but before the Notes app has been opened a first time
• The notes draft are shared with users with edit permission. During migration for the notes with existing drafts, we will keep the latest edited draft only and share it.
• The news draft are shared with users with edit permission. During migration for the news with existing drafts, we will keep the latest edited draft only and share it.

Update :
The following requirement are moved in another MIP #130 :

• migration activities
• sheduled publication
• following filters in news app : posted articles, published articles, my posted articles, archived articles, scheduled articles
• in order to hide the options scheduled publication and filters, a flag is expected

2. Technical Requirements

Feature Flags

Property Name	Default Value	Target Audience	Functional Behaviour
newsScheduleAndFilterDisplaying	false	users	enable/disable new scheduling and filters in news app except draft filter

3. Software Architecture

Services & processing

Social addon

• A new method getMetadataItemsByMetadataNameAndTypeAndObjectAndSpaceIds need to be implemented for metadata service layer
• A new method getMetadataItemsByFilter need to be implemented for metadata API layers.

Notes addon

• Currently only the "Home" note page can be created at the wiki space root level and any creation of another page at the same level updates the "Home" note page. In order to create a dedicated "news" note page at the wiki space root level, we need to allow the creation of any page at the wiki space root level by updating the method createPage of JPADataStorage layer (https://github.com/Meeds-io/notes/blob/develop/notes-service/src/main/java/org/exoplatform/wiki/jpa/JPADataStorage.java#L217).
• A new method removeDraftById need to be implemented for notes API layers.
• In ordre to implement the shared notes draft managment, we need to implement get draft methods without username for notes API layers

Content addon

• A new meeds addon called content will be implemented:
  • The rest and service layers of "Content" addon will be the same as eXo "News" addon rest and service layers.
  • The jcr storage layer will be removed.
  • All the jcr/ecms dependencies will be removed.
  • The Content service layer will call the Notes service layer and the Metadata service layer.

Sequence diagram for create draft article use case:

Sequence diagram for get draft article use case

Data and persistence

• A new metadataType "news" should be configured using org.exoplatform.social.metadata.MetadataTypePlugin component plugin.

    <component-plugin>
      <name>NotesMetadataPlugin</name>
      <set-method>addMetadataTypePlugin</set-method>
      <type>org.exoplatform.social.metadata.MetadataTypePlugin</type>
      <init-params>
        <value-param>
          <name>shareable</name>
          <value>true</value>
        </value-param>
        <value-param>
          <name>allowMultipleItemsPerObject</name>
          <value>true</value>
        </value-param>
        <object-param>
          <name>metadataType</name>
          <object type="org.exoplatform.social.metadata.model.MetadataType">
            <field name="id">
              <int>1000</int>
            </field>
            <field name="name">
              <string>news</string>
            </field>
          </object>
        </object-param>
      </init-params>
    </component-plugin>

• A new metadata "news" associated to the metadataType "news" detailed above should be created into METADATA table using the configuration of org.exoplatform.social.metadata.MetadataInitPlugin component plugin. This metadata will be used in order to complete the WIKI_DRAFT_PAGES, WIKI_PAGE_VERSIONS or WIKI_PAGES fields with news additional fields.

    <component-plugin>
      <name>DraftNewsMetadataPlugin</name>
      <set-method>addMetadataPlugin</set-method>
      <type>org.exoplatform.social.metadata.MetadataInitPlugin</type>
      <init-params>
        <object-param>
          <name>metadata</name>
          <object type="org.exoplatform.social.metadata.model.Metadata">
            <field name="type">
              <object type="org.exoplatform.social.metadata.model.MetadataType">
                <field name="id">
                  <int>1000</int>
                </field>
                <field name="name">
                  <string>news</string>
                </field>
              </object>
            </field>
            <field name="name">
              <string>news</string>
            </field>
          </object>
        </object-param>
      </init-params>
    </component-plugin>

SOC_METADATAS table:

METADATA_ID	TYPE	Name	.....
5	1000	News	...........

• When a news draft article, a news article or a news article version is created, a metadata item should be created into SOC_METADATA_ITEMS table. This metadata item will have:
  • The "news" metadata id as metadata_id
  • "draftPage", "page" or "pageVersion" as object_type, this will ensure to distinguish between metadata items with the same OBJECT_ID
  • The corresponding notes draft_page_id, page_id or page_version_id as OBJECT_ID.

METADATA_ITEM table:

METADATA_ITEM_ID	METADATA_ID	OBJECT_TYPE	OBJECT_ID	PARENT_OBJECT_ID	...........
100	5	newsDraftPage	100	null	...........
101	5	newsPage	101	null	...........
102	5	newsPageVersion	102	null	...........
103	5	newsLatestDraftPage	103	101	...........

• For the newly created metadata item, some properties should be created into METADATA_ITEM_PROPERTIES table, which will correspond to the news additional fields. These properties are already known and will be created at runtime:
  • Specific properties for the metadata items of OBJECT_TYPE "newsPageVersion", "newsLatestDraftPage" or "newsDraftPage" since they will contain informations related to a draft article, the latest draft article or a specific article version:
    • illustrationId: The file id of the article banner, this property value will be specific to the language of the notes draft page, the notes page or the notes page version.
    • uploadId: The upload id of the article banner.
    • summary: The article summary, this property value will be specific to the language of the notes draft page, the notes page or the notes page version.
    • attachments_ids: The jcr uuids of the last article version attachments.
  • A specific property for metadata items of OBJECT_TYPE "newsLatestDraftPage":
    • activityPosted: A boolean value which indicates if the corresponding generated activity of the lastest draft articleis posted in the space activity stream.
  • Specific properties for only metadata items of OBJECT_TYPE "newsPage" since they will contain informations related to the last article version and not those of a draft article, a latest draft article or a specific article version:
    • published: A boolean value which indicates if the last article version is published.
    • activityPosted: A boolean value which indicates if the corresponding generated activity of the last article version is posted in the space activity stream.
    • publicationState: The current publication state of the last article version.
    • scheduled: A boolean value which indicates if the last article version is scheduled.
    • schedulePostDate: The schedule date of the last article version.
    • activities: The shared activities of the last article version activity.
    • audience: The publish audience of the last article version.
    • viewsCount: The views count of the last article version.
    • viewers: The viewers user ids of the last article version.

METADATA_ITEM_PROPERTIES table:

METADATA_ITEM_ID	NAME	VALUE
100	IllustrationId	100
100	summary	My draft article Summary
101	activityPosted	true
101	published	true
101	schedulePostDate	My article version 20 Summary
102	IllustrationId	100
102	summary	My article first version Summary
103	IllustrationId	100
103	summary	My draft for existing article Summary
103	activityPosted	true
.....

[azayati]

I have created this DRAFT Mip with the content discussed during our Meeting of January 23rd.
@boubaker Can you take a look, give me your first feedbacks/remarks in order to start discussions ?
cc @rdenarie @margondicco @srenault-meeds

[plamarque]

Added a rationale

[boubaker]

IMHO the Metadata API is not the good solution to use here.
In fact, we are designing a new Data Model for a new feature which will reuse the notes editor, but not the data model of notes as it is.
The Metadata API is designed to decorate mixed types of data, such as adding favorites on spaces, users, notes, activities ...
Using it to design a new data model on top of an existing data Model can be used as a trick but not for design a new addon in product.

In addition, polluting the Notes data model with News fields and Business logic can make both products unmaintainable and hard to evolve. I propose to study more the Software Architecture by bringing a second option where we use a specific database Model for News (duplicating code in this special case, can reduce complexity and reduces the cost of flexibility IMO). Eager to discuss more about this.

[azayati]

@boubaker If I have understand your point, you mean that News should have its own tables (exactly like notes ones) + the specific news fields. A duplication of code will be done also for all notes layers + search.
In that case IMO, no relation between a news entity and a notes entity, we are no more talking about a migration of news data model to notes one, but it will be an upgrade of news data model from jcr to jpa.

cc @rdenarie @margondicco

[boubaker]

@boubaker If I have understand your point, you mean that News should have its own tables (exactly like notes ones) + the specific news fields. A duplication of code will be done also for all notes layers + search. In that case IMO, no relation between a news entity and a notes entity, we are no more talking about a migration of news data model to notes one, but it will be an upgrade of news data model from jcr to jpa.

cc @rdenarie @margondicco

For my part I think that we have the following disadvantages with current design:

• The News table fields are intrusive inside Notes Data Model which shouldn't change to support News, but News has to extend Notes to add more information
• Changing Data Model and business logic of notes would add complexity to the current behavior of notes
• Changing Data Model and business logic of notes would add instability to an existing addon which is quite stable
• Adding an addon business logic inside another addon which doesn't use it, makes the maintainability and evolutivity quite hard to do

To design an addon which will extend another (like we do in all other type of addons), we have to add extension points (using extension registry for the front end And Plugins for the backend). Considering this, IMO, we should rethink the technical design to benefit from Notes existing behaviors instead. There is no functional requirement in the MIP, thus it will be difficult to design a good solution. But if :

• we aim to be able to switch a Note to a News
• we aim to reuse the Notes versioning system
• we aim to add on top of the Notes Page some additional fields when it's switched to a News
• we aim to reuse the Notes editor, with additional editable fields when it's a news type

Then, like we do for Activities, we should keep the original behavior of Notes without changing it and just add a Data Model Table to decorate the Notes Object with additional fields like:

• Summary
• Title
• Published Version
• Publisher
• Publication Date
  ...

Those information should be stored and managed outside of Notes and managed in UI using extensionRegistry extensibility.

PS: as the current technical requirement doesn't use Metadata API for adding additional fields (like Summary & Title), I think it would be better to not use it for News for now and keep the design of using Metadata Model when a dedicated MIP to CMS is proposed. (It would simplify the development of current MIP, but I let you choose between using a dedicated Table for News or Using the Metadata API)

[azayati]

I vote for using dedicated tables for news and not the metadata, since the data model of news will complete the "Notes" one, we should have:

• Three tables like for notes (NEWS, NEWS_VERSIONS, NEWS_DRAFT) having the id of the corresponding note object.
• Should we add a type field in "Notes" tables in order to retrieve the stored notes (with type "News") ? or we retrieve them based on the notes ids referenced from the "News" tables ?

[boubaker]

Should we add a type field in "Notes" tables in order to retrieve the stored notes (with type "News") ? or we retrieve them based on the notes ids referenced from the "News" tables ?

I let you decide, from my part not changing notes but making it extensible to support News features through extensionRegistry & plugins is sufficient for me.