E is a vocal assistant designed to be respectful of user privacy over theirs data.
Elivia is a vocal assistant designed to be respectful of user privacy over theirs data.
The goal of this project is to be seen incorporated in /e/ os, a product from the [e foundation](https://e.foundation/), do to so it is supervised by [Gaël Duval](https://fr.wikipedia.org/wiki/Ga%C3%ABl_Duval).
E Vocal Assistant is composed of 3 parts, a front end built in Kotlin using gradle, a back-end generously inspired from [Olivia](https://github.com/olivia-ai/olivia)'s design, packaging an AI, and a CLI tool made for testing purpose.
**Packages** are the **Elivia**'s addon they will make your life easier when contributing to thee application and are totally independent from the rest of **Elivia**
Each **Package** holds several **Modules** and each module have its own trigger and response.
In the future, modules from the same package will share the same data.
## Format
Package all have the same architecture tree
first you have to create a folder ***package_name*** inside the *package* folder
inside this, you will have
* Your go file and architecture that **must** build.
* A *res* folder
The res folder will hold a *locales* folder itself containing a folder for each of the languages (we are currently supporting English -tagged **en**- and French -tagged **fr**- )
each of these language folder will hold a **response.json** and a **triggers.json**
## Code Architecture
For the moment only Go is supported, we may add a support for all language compiling to ELF.
Note that you can update the [loader](https://github.com/PoCFrance/e/blob/master/back/plugins/package.go) function to change the way the program load functions from external libraries
Each Module will need an entry point, it been a function with this exact prototype
In the **sentence** variable you will find the request that triggered your module, the **entry** will be explained inside the **Triggers** part
The Return type has two things.
First : the **Tag**, it will choose a random response from the one defined in the `response.json`
second : the **Map** you will be able to feed all the information you want to transfer to you front end, there is only one type of data you have to feed, and we will explain it in the **Response** part
## Triggers
This is where the majority of the configuration is. it define when and how the Module will be called. as well as the module itself
it is recommended to check the [sms](https://github.com/PoCFrance/e/blob/master/back/package/sms/res/locales/en/triggers.json) package as it serve as example.
In this json, you will have an Array of Modules.
Each module is defined by an **Array** of **Module Type** thus, a module can be triggered by multiple ways, and have various catchphrase without having issues to define them all.
A **Module Type** has these fields :
***entries** - **Entry Type Array** - A possibly null field containing all the resources for extracting meaning from the sentence (i.e. text to send and contact inside the sms module) **entries** can be found inside the module's function when call, in the form of a map.
***pattern** - **String Array** - Key word needed to match this module
***olivia-feed** - **String Array** - Real life full sentence serving as example for the current IA, will be deprecated when the IA will be reworked
We will also define the **Entry Type** straight away:
***name** - **String** - The name of the entry (the one you will have to query in the map)
***parser** - **String** - There is 3 parsers existing, **after**, **before**, and **between ** depending on which type of data you want to retrieve.
***resources** - Custom Object - depends on the parser type
* The after and before will need the **key** - **String** - and the **x** - **Int** - field, the first being the the match from which you extract and the second been the number of occurrences you skip before counting it a match (note you can have negative number, thus `"x": -1` will lead to seek the last occurrences)
* The between parser will need two - **String** -, **after** and **before** and two - **Int** -, **x** and **y**, and will get all the text matching between the two field
It works as if you had a union between the after parser with the **after** and **x** value, and a before parser with the **before** and **y** value.
To summarize it all, there is a little example of a valid triggers.json file
```json
[
{
"FunctionName/Module":[
{
"entries":[
{
"name":"",
"parser":"",
"ressources":{}
}
],
"pattern":[
"-"
],
"olivia-feed":[
"- - - -",
"- - - - -"
]
}
]
}
]
```
## Response
`Response.json` is pretty straight forward, you have a list of Tag - **String** -, associated with message - **String Array** -
When you return from your Go function, you have to specify a Tag, it will serve to pick one of the messages array, and will pick a answer at random
Messages aren't totally basic string, they can be formatted with data from your code.
To do so, you have to precise which part have to be filled with this marking `%` before and after (i.e. : `hello, %variable%` ).
remember in the code part when we said we would come back on the map, this is precisely the other use case we have of it
When returning with a tag implementing formatted variable, you will have to add a field in the map with the text to replace with.
for example, the `hello, %variable%` will need you to return something like this
the "pattern", "feed", and "callback" may seems straight forward, yet, the entries needs a bit more explainations
As said right before, the main part about Elia, is the IA, it was extracted from [Olivia's](https://github.com/olivia-ai/the-math-behind-a-neural-network). For all documentation about the way it works, we strongly recommend checking their repository.
### Packages
### Entries
One thing we added upon it, is the **package** system, Elia is able to load, compile, and train package at run time allowing you to add, remove or edit response at run time.
To say it briefly, its the bread and butter of Elia, you can format response and trigger, call your own function, and safely extract data from the sentences that triggered your package.
Entry helps to retreive element from the string you match, note that you should leave a blank in the pattern if you want to catch something there.
**Packages** seems complicated but they aren't that much of deal, really. Plus there is a full documentation on how to write them, which features are implemented and the best practices around them.
entries are composed of 3 elements
### Websocket
```json
{
"name":""
"parser":""
"resources":{}
}
```
Elia uses Websocket to communicate with the front-end, the only route used is the serve one, allowing you to handle a sentence through the IA.
the name is string you will use to retreive that entry within your code.
the parser is the function that will be called to find out your entry, 3 of them are available nowadays :
## What's next ?
* after : gets the rest of the string after the match
it uses a specific ressourse to define the match requieres
Elia is a PoC project, so the focus may change a bitt depending of the team taking over the project in the next months. but the main features left to implement are
* before : works the same as after but gets everything before, match not included
* between : gets everything between the 2 matchs
it also define a resource format to match its need
```json
"ressources":{
"after":"",//Thekeyforthestartingmatch
"x":0,//thexforthestartingmatch
"before":"",//thekeyforthestartingmatch
"y":0//thexforthestartingmatch
}
```
* Enhance the IA - The current one works well, but with a **Text Data Vectorization** algorithm extracting meaning could have better result thus, leading to less misunderstanding error
* Make it a run time - I lied when I said it was a run time interpreter of packages. It does, indeed, compile and run every package at run time, but it doesn't reload itself yet. This is an easy update that would allow a permanently running soft without any restart needed.
* Package data - We would like to implement a tool making user's data manipulation easy inside of package, thus, you would be able to remember previous action from the user and act upon them.
* Packages, packages and again, packages - We want to add enough of them so user would be able to choose which one they want and load only the one needed when making request with a user thus having a "store" of packages from which user would build their own little IA.
No newline at end of file
As you can see, we have plenty of ideas and this is only a small grasp of what we plan to do.
