Donate to e Foundation | Murena handsets with /e/OS | Own a part of Murena! Learn more

Unverified Commit 5ed2351d authored by Wolf-Martell Montwé's avatar Wolf-Martell Montwé Committed by GitHub
Browse files

Merge pull request #9947 from wmontwe/chore/9938/add-gradle-task-to-update-demo-backend 

chore(demo): add gradle task to update demo backend
parents cf339151 55eb7b97

backend/demo/README.md

0 → 100644
+78 −0
Original line number Diff line number Diff line 
# Demo Backend



This module provides a self‑contained, offline backend implementation used by the app to showcase and manually test

email UI and flows without connecting to a real mail server. It implements the Backend API and loads demo data from

resources bundled with the library.



## What it does



- Exposes a Backend that

  - returns a predefined folder list

  - supports basic sync of message lists

  - supports threaded conversations (based on standard Message-Id, In-Reply-To, and References headers)

  - pretends to move/copy/upload messages successfully

  - sends messages by handing them to the app storage layer (no network)

- Loads folders and messages from `src/main/resources/mailbox`.



## How data is organized



- Folder tree definition: `src/main/resources/mailbox/contents.json`

  - Describes folders by `serverId`, display name, type (INBOX, SENT, …), and a list of `messageServerIds` per folder.

  - Supports nested folders through the `subFolders` field. The backend flattens nested folders internally so they show up as "Parent/Child" names.

  - Special folders (Inbox, Drafts, Sent, Spam, Trash, Archive) are ensured to always exist.

- Messages: EML files in `src/main/resources/mailbox/<folderServerId>/<messageServerId>.eml`

  - Example: `src/main/resources/mailbox/inbox/intro.eml` corresponds to `folderServerId=inbox` and `messageServerId=intro`.



Key classes



- DemoBackend: Backend implementation wired to simple commands.

- DemoStore: In‑memory source of truth backed by resources.

- DemoDataLoader: Reads contents.json and parses .eml files into Message objects.



Limitations (by design)



- No real network access; search, part fetching, and some operations are not implemented and will throw.

- Push is not supported.

- Only data found in contents.json and the matching .eml files is available.



## Using the demo backend in apps



This module is a Kotlin/JVM library. Applications can depend on `backend:demo` and select the demo backend when creating

accounts for testing/development. The exact wiring is app‑specific (see the app modules in this repository for how

they register/select backends).



## Editing demo content



1) Add a new message

- Place your EML file at: `src/main/resources/mailbox/<folderServerId>/<yourMessageId>.eml`

- Run `./gradlew :backend:demo:updateDemoMailbox` to regenerate `src/main/resources/mailbox/contents.json`.



2) Add a new folder (optionally nested)

- Create the corresponding directory under `src/main/resources/mailbox/<yourFolderServerId>/` and place the `.eml` files inside it.

- Then run `./gradlew :backend:demo:updateDemoMailbox` to update `contents.json`.



### Threaded messages



Threading is supported by the app when messages include standard headers. The demo backend simply exposes the messages; the UI groups them into threads.



To create a conversation thread in a folder:



- Give each message a unique `Message-Id` header.

- For replies, set `In-Reply-To` to the `Message-Id` of the parent message.

- Maintain a `References` header that contains the chain of ancestor `Message-Id`s (root first, then each reply). Many clients do this automatically; for demo EMLs, edit the headers manually.

- Place all messages of a thread in the same folder.

- Subject prefixes like `Re:` are optional and not used for threading.



Example headers in a reply message:



```

Message-Id: <reply-2@example.test>

In-Reply-To: <root-1@example.test>

References: <root-1@example.test>

```



Notes and limitations for threads:



- Cross-folder threading is not supported by the demo backend; keep a thread’s messages in one folder.

- The backend does not infer threads from filenames or `messageServerIds`; only the MIME headers control threading.

backend/demo/build.gradle.kts

+110 −0
Original line number Diff line number Diff line 
import groovy.json.JsonOutput



plugins {

    id(ThunderbirdPlugins.Library.jvm)

    alias(libs.plugins.android.lint)
@@ -12,3 +14,111 @@ dependencies { 


    testImplementation(projects.mail.testing)

}



tasks.register<UpdateDemoMailbox>("updateDemoMailbox") {

    group = "demo"

    description = "Update mailbox/contents.json from src/main/resources/mailbox contents."



    inputDir.set(layout.projectDirectory.dir("src/main/resources/mailbox"))

    outputFile.set(layout.projectDirectory.file("src/main/resources/mailbox/contents.json"))

}



@CacheableTask

abstract class UpdateDemoMailbox : DefaultTask() {



    @get:InputDirectory

    @get:PathSensitive(PathSensitivity.RELATIVE)

    abstract val inputDir: DirectoryProperty



    @get:OutputFile

    abstract val outputFile: RegularFileProperty



    @TaskAction

    fun generate() {

        val mailboxRoot = inputDir.get().asFile



        val topLevelFolderMap = mailboxRoot.listFiles()

            ?.filter { it.isDirectory }

            ?.sortedBy { it.name }

            ?.mapNotNull { folder ->

                buildFolderNode(folder).takeIf { it.first }?.let { folder.name to it.second }

            }

            ?.toMap(LinkedHashMap()) ?: linkedMapOf()



        // Ensure special folders exist (even if empty)

        SPECIAL_FOLDERS.forEach { id ->

            topLevelFolderMap.putIfAbsent(

                id,

                linkedMapOf(

                    "name" to displayName(id),

                    "type" to folderType(id),

                    "messageServerIds" to emptyList<String>(),

                ),

            )

        }



        // Reorder: special folders first, then others alphabetically

        val orderedTopLevelMap = linkedMapOf<String, Map<String, Any?>>()

        SPECIAL_FOLDERS.forEach { id -> topLevelFolderMap[id]?.let { orderedTopLevelMap[id] = it } }

        topLevelFolderMap.keys.filter { it !in SPECIAL_FOLDERS }.sorted().forEach { id ->

            orderedTopLevelMap[id] = topLevelFolderMap[id]!!

        }



        val contentsFile = outputFile.get().asFile

        contentsFile.parentFile?.mkdirs()



        val json = JsonOutput.prettyPrint(JsonOutput.toJson(orderedTopLevelMap))

        contentsFile.writeText("$json\n")

        println("Wrote \"${contentsFile.toPath()}\" with ${orderedTopLevelMap.size} top-level folder(s)")

    }



    private fun buildFolderNode(folder: File): Pair<Boolean, Map<String, Any?>> {

        val files = folder.listFiles() ?: emptyArray()

        val messageIds = files.filter { it.isFile && it.name.endsWith(".eml") }

            .map { it.name.removeSuffix(".eml") }

            .sorted()



        val subFolderNodes = files.filter { it.isDirectory }

            .sortedBy { it.name }

            .mapNotNull { subFolder ->

                buildFolderNode(subFolder).takeIf { it.first }?.let { subFolder.name to it.second }

            }

            .toMap(LinkedHashMap())



        val shouldInclude = messageIds.isNotEmpty() || subFolderNodes.isNotEmpty() || isSpecialFolder(folder.name)



        val node = linkedMapOf<String, Any?>(

            "name" to displayName(folder.name),

            "type" to folderType(folder.name),

            "messageServerIds" to messageIds,

        )

        if (subFolderNodes.isNotEmpty()) {

            node["subFolders"] = subFolderNodes

        }

        return shouldInclude to node

    }



    private fun isSpecialFolder(name: String) = SPECIAL_FOLDERS.contains(name.lowercase())



    private fun folderType(name: String) = when (name.lowercase()) {

        "inbox" -> "INBOX"

        "drafts" -> "DRAFTS"

        "sent" -> "SENT"

        "spam" -> "SPAM"

        "trash" -> "TRASH"

        "archive" -> "ARCHIVE"

        else -> "REGULAR"

    }



    private fun displayName(name: String): String {

        return name.replace('-', ' ')

            .replace('_', ' ')

            .split(' ')

            .filter { it.isNotBlank() }

            .joinToString(" ") { it.lowercase().replaceFirstChar { c -> c.titlecase() } }

    }



    companion object {

        private val SPECIAL_FOLDERS = listOf("inbox", "drafts", "sent", "spam", "trash", "archive")

    }

}

backend/demo/src/main/kotlin/app/k9mail/backend/demo/DemoDataLoader.kt

+2 −2
Original line number Diff line number Diff line
@@ -11,13 +11,13 @@ internal class DemoDataLoader { 


    @OptIn(ExperimentalSerializationApi::class)

    fun loadFolders(): DemoFolders {

        return getResourceAsStream("/contents.json").use { inputStream ->

        return getResourceAsStream("/mailbox/contents.json").use { inputStream ->

            Json.decodeFromStream<DemoFolders>(inputStream)

        }

    }



    fun loadMessage(folderServerId: String, messageServerId: String): Message {

        return getResourceAsStream("/$folderServerId/$messageServerId.eml").use { inputStream ->

        return getResourceAsStream("/mailbox/$folderServerId/$messageServerId.eml").use { inputStream ->

            MimeMessage.parseMimeMessage(inputStream, false).apply {

                uid = messageServerId

            }

backend/demo/src/main/resources/contents.json

deleted100644 → 0
+0 −88
Original line number Diff line number Diff line 
{

  "inbox": {

    "name": "Inbox",

    "type": "INBOX",

    "messageServerIds": [

      "intro",

      "many_recipients",

      "thread_1",

      "thread_2",

      "inline_image_data_uri",

      "inline_image_attachment",

      "localpart_exceeds_length_limit"

    ]

  },

  "trash": {

    "name": "Trash",

    "type": "TRASH",

    "messageServerIds": []

  },

  "drafts": {

    "name": "Drafts",

    "type": "DRAFTS",

    "messageServerIds": []

  },

  "sent": {

    "name": "Sent",

    "type": "SENT",

    "messageServerIds": []

  },

  "archive": {

    "name": "Archive",

    "type": "ARCHIVE",

    "messageServerIds": []

  },

  "spam": {

    "name": "Spam",

    "type": "SPAM",

    "messageServerIds": []

  },

  "turing": {

    "name": "Turing Awards",

    "type": "REGULAR",

    "messageServerIds": [

      "turing_award_1966",

      "turing_award_1967",

      "turing_award_1968",

      "turing_award_1970",

      "turing_award_1971",

      "turing_award_1972",

      "turing_award_1975",

      "turing_award_1977",

      "turing_award_1978",

      "turing_award_1979",

      "turing_award_1981",

      "turing_award_1983",

      "turing_award_1987",

      "turing_award_1991",

      "turing_award_1996"

    ]

  },

  "nested": {

    "name": "Nested",

    "type": "REGULAR",

    "messageServerIds": [

      "nested_1"

    ],

    "subFolders": {

      "nested_level_1": {

        "name": "Nested Level 1",

        "type": "REGULAR",

        "messageServerIds": [

          "nested_level_1_1",

          "nested_level_1_2"

        ],

        "subFolders": {

          "nested_level_2": {

            "name": "Nested Level 2",

            "type": "REGULAR",

            "messageServerIds": [

              "nested_level_2_1",

              "nested_level_2_2"

            ]

          }

        }

      }

    }

  }

}

backend/demo/src/main/resources/mailbox/contents.json

0 → 100644
+98 −0
Original line number Diff line number Diff line 
{

    "inbox": {

        "name": "Inbox",

        "type": "INBOX",

        "messageServerIds": [

            "01-intro",

            "02-many-recipients",

            "03-thread-1",

            "04-thread-2",

            "05-inline-image-data-uri",

            "06-inline-image-attachment",

            "07-localpart-exceeds-length-limit"

        ]

    },

    "drafts": {

        "name": "Drafts",

        "type": "DRAFTS",

        "messageServerIds": [

            

        ]

    },

    "sent": {

        "name": "Sent",

        "type": "SENT",

        "messageServerIds": [

            

        ]

    },

    "spam": {

        "name": "Spam",

        "type": "SPAM",

        "messageServerIds": [

            

        ]

    },

    "trash": {

        "name": "Trash",

        "type": "TRASH",

        "messageServerIds": [

            

        ]

    },

    "archive": {

        "name": "Archive",

        "type": "ARCHIVE",

        "messageServerIds": [

            

        ]

    },

    "nested": {

        "name": "Nested",

        "type": "REGULAR",

        "messageServerIds": [

            "nested_1"

        ],

        "subFolders": {

            "nested_level_1": {

                "name": "Nested Level 1",

                "type": "REGULAR",

                "messageServerIds": [

                    "nested_level_1_1",

                    "nested_level_1_2"

                ],

                "subFolders": {

                    "nested_level_2": {

                        "name": "Nested Level 2",

                        "type": "REGULAR",

                        "messageServerIds": [

                            "nested_level_2_1",

                            "nested_level_2_2"

                        ]

                    }

                }

            }

        }

    },

    "turing": {

        "name": "Turing",

        "type": "REGULAR",

        "messageServerIds": [

            "turing_award_1966",

            "turing_award_1967",

            "turing_award_1968",

            "turing_award_1970",

            "turing_award_1971",

            "turing_award_1972",

            "turing_award_1975",

            "turing_award_1977",

            "turing_award_1978",

            "turing_award_1979",

            "turing_award_1981",

            "turing_award_1983",

            "turing_award_1987",

            "turing_award_1991",

            "turing_award_1996"

        ]

    }

}