sel/wgu
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases 1 Packages Wiki Activity Actions

first commit

selene 2025-11-24 23:18:50 +01:00
commit 2d93b0e984
Signed by: sel
SSH key fingerprint: SHA256:33R/4Rx5Lu4o81LyJyXdMrmP5CJ6j7j1Soo0Dn7mKc0
6 changed files with 314 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

3
Home.md Executable file

@ -0,0 +1,3 @@
# welcome to the wiki
for an overview of what wgu is, see the [readme](https://codeberg.org/selstacker/wgu)

3
_Sidebar.md Executable file

@ -0,0 +1,3 @@
- [objects](object.md)
- [variants of `ModuleItem`](modularity.md)
- [built-in modules](modules.md)

17
meta-connection.md Normal file

@ -0,0 +1,17 @@
# `meta/connection`
`meta/connection` is an object template of the `meta` module used for representing connections & relationships between object instances.
## template
### inputs
| name | transforms | subobjects | conditions | duplicates | value |
| ---------- | ----------------- | ---------- | --------------------------------------------------------------- | ---------- | ----------------------------------------------------------------- |
| `variant` | `meta/text:value` | none | `input.variant`: `function`, `transform`, `subobject`, `custom` | false | for specifying the variant of connection for other properties |
| `from` | `meta/text:value` | none | none | true | for specifying the reference string of the **source object** |
| `to` | `meta/text:value` | none | none | true | for specifying the reference string of the **destination object** |
| `property` | `meta/text:value` | none | `input.variant`: `transform`, `subobject` | false | for specifying the affected property in transforms and subobjects |
| `value` | `meta/text:value` | none | `input.variant`: `custom` | false | for specifying the note in custom connections |

33
modularity.md Executable file

@ -0,0 +1,33 @@
# modularity
> 🚧 **under construction**
>
> **this document needs more expansion!** you can contribute by documenting how this feature or module operates in the latest available version (also check for pull requests that would change that)
## `... ModuleItem::Template`
> you're probably looking for
an object template is a [toml](https://toml.io) blueprint of an `wgu::ObjectInstance`. it is exported from modules as a `&'static str`.
for referencing templates, the name of the module and the name of the template are used, separated by a `/` (slash) symbol: `{object_type}`
## `... ModuleItem::Validator`
validators take one [object instance](object.md#instances), and output a `Result` based on whether the object instance passes validation.
for referencing validators, the following syntax is used: `{object_type}:func:validator`
## `... ModuleItem::Calculator`
calculators take one [object instance](object.md#instances), and output one object instance, which *should* be of the same `object_type`.
calculators are used by wgu to modify an object instance, most often by populating properties under `local` and `remote`.
for referencing calculators, the following syntax is used: `{object_type}:func:{name}`
## `... ModuleItem::Function`
functions take hashmaps of named object instances as inputs (and parameters), and output another hashmap of named object instances.
for referencing functions, the following syntax is used: `func:{name}`

89
modules.md Executable file

@ -0,0 +1,89 @@
# built-in modules
## `meta`
this module has object templates for primitives and other basic types of data most other modules rely on.
### general data types
| object template | readable name |
| --------------------------------------- | ------------- |
| `meta/bool` | boolean |
| [`meta/connection`](meta-connection.md) | connection |
| `meta/dummy` | dummy text |
| `meta/log` | log |
| `meta/number` | number |
| `meta/text` | text |
| `meta/tag` | tag |
| `meta/timestamp` | timestamp |
### image files
| object template | readable name |
| ---------------------- | --------------- |
| `meta/file.image.png` | png image file |
| `meta/file.image.jpg` | jpeg image file |
| `meta/file.image.gif` | gif image file |
| `meta/file.image.webp` | webp image file |
### audio files
| object template | readable name |
| ---------------------- | --------------- |
| `meta/file.audio.mp3` | mp3 audio file |
| `meta/file.audio.wav` | wav audio file |
| `meta/file.audio.flac` | flac audio file |
| `meta/file.audio.ogg` | ogg video file |
| `meta/file.audio.opus` | opus video file |
### video files
| object template | readable name |
| ---------------------- | --------------- |
| `meta/file.media.mp4` | mp4 media file |
| `meta/file.media.webm` | webm media file |
| `meta/file.media.mkv` | mkv media file |
| `meta/file.media.avi` | avi media file |
| `meta/file.media.ts` | ts media file |
### document files
| object template | readable name |
| ------------------------- | --------------------------------- |
| `meta/file.document.docx` | microsoft word document |
| `meta/file.document.xlsx` | microsoft excel spreadsheet |
| `meta/file.document.pptx` | microsoft powerpoint presentation |
| `meta/file.document.odt` | opendocument text |
| `meta/file.document.ods` | opendocument spreadsheet |
| `meta/file.document.epub` | epub e-book |
| `meta/file.document.txt` | plain text file |
| `meta/file.document.md` | markdown document |
| `meta/file.document.pdf` | pdf document |
| `meta/file.document.rtf` | rich text format document |
### binaries & executables
| object template | readable name |
| ---------------------- | ---------------------------- |
| `meta/file.binary.exe` | windows executable |
| `meta/file.binary.dll` | dynamic link library |
| `meta/file.binary.ps1` | powershell script |
| `meta/file.binary.bat` | windows batch script |
| `meta/file.binary.sh` | unix shell script |
| `meta/file.binary.elf` | elf binary |
| `meta/file.binary.jar` | java archive |
| `meta/file.binary.apk` | android package |
| `meta/file.binary.iso` | disk image |
| `meta/file.binary.bin` | raw binary |
### archives & compression
| object template | readable name |
| ----------------------- | ------------- |
| `meta/file.archive.zip` | zip archive |
| `meta/file.archive.rar` | rar archive |
| `meta/file.archive.7z` | 7zip archive |
| `meta/file.archive.tar` | tar archive |
| `meta/file.archive.gz` | gzip archive |
| `meta/file.archive.bz2` | bzip2 archive |
| `meta/file.archive.xz` | xz archive |

169
object.md Executable file

@ -0,0 +1,169 @@
# objects
**the main unit of organization within the modular object format is the object instance.** an object instance (`wgu::ObjectInstance`, stored & transmitted as a [json object literal](https://benalman.com/news/2010/03/theres-no-such-thing-as-a-json/)) is a data structure representing a piece of computer data of a specified type.
core object data is categorized into three categories, `input`, `local`, `remote`. the structure of these categories depends on the object instance's `object_type`. due to this, within wgu, those categories are stored as *json object strings* in the corresponding columns of the `objects` sqlite table. object metadata (`variant`, `object_type`, timestamps, hashes, etc.) are stored alongside these categories.
the keys in object instances are generally referred to as **properties**.
> ⚠️ **warning**
>
> a **big** pitfall early on is confusing the properties `variant` with `input.variant`
>
> `variant` is for differentiating between multiple object instances with the same `input_sha256`, while `input.variant` is most often used for defining multiple structures inside one object, when it can represent multiple things (the best example of that is `meta/connection`)
for referencing object instances, the properties `input_sha256` and `variant` are used, separated by a `+` (plus) symbol. to refer to a property within an object:
- add a `:` (colon) symbol
- followed by the category and a `.` (dot) (`input` | `local` | `remote`, unless you're referring to a transform destination, in which case this *must* be ommited as it's always going to be `input`)
- followed by the property name
> 🧪 **example:** `5aea...b0ca+0:input.value`
if the property you're referring to can be [duplicated](#duplicates), you also have to add the position of the value in the resulting array within `[]` (square brackets):
> 🧪 **example:** `acab...1312+0:remote.some.thing[2]`
## instances
below are the columns of the table `objects` in wgu's sqlite database. these categories and properties exist for every object.
### categories
| category | transforms | subobjects | conditions | duplicates | purpose |
| -------- | ---------- | ---------- | ---------- | ---------- | ------------------------------------------------------------------ |
| `input` | none | none | none | false | holds data needed for populating properties under other categories |
| `local` | none | none | none | false | holds locally calculated data related to the object instance |
| `remote` | none | none | none | false | holds remotely obtained data related to the object instance |
| `logs` | none | none | none | true | logs produced by functions |
### general
| property | transforms | subobjects | conditions | duplicates | value |
| ------------- | ------------------- | ---------- | ---------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `variant` | `meta/number:value` | none | none | false | for differentiating between multiple objects with the same `hashes.input_sha256`. starts at `0` (zero) for each hash |
| `object_type` | `meta/text:value` | none | none | false | for specifying the object's expected content. the string before the `/` (slash) is the module name, after that `.` (dots) are used to separate layers of nesting |
| `plotted` | `meta/bool:value` | none | none | false | **helper property for wgu.** for specifying if the object is plotted to the main graph view |
### timestamps
> 🚧 **under construction**
>
> **this section needs more expansion!** you can contribute by documenting how this feature or module operates in the latest available version (also check for pull requests that would change that)
| property | transforms | subobjects | conditions | duplicates | value |
| --------- | ------------------------ | ---------- | ---------- | ---------- | -------------------------------------------- |
| `created` | `meta/timestamp:unix-ms` | none | none | false | timestamp of the object's creation |
| `edited` | `meta/timestamp:unix-ms` | none | none | false | timestamp of the object's last manual edit |
| `hashed` | `meta/timestamp:unix-ms` | none | none | false | timestamp of `input_sha256` being calculated |
### hashing
| property | transforms | subobjects | conditions | duplicates | value |
| -------------- | ----------------- | ---------- | ---------- | ---------- | --------------------------------------------------------------------------------------- |
| `input_sha256` | `meta/text:value` | none | none | false | for naming/querying/referencing objects (together with `variant`) |
| `full_sha256` | `meta/text:value` | none | none | false | for validating/verifying/auditing objects (together with properties under `timestamps`) |
`input_sha256` is calculated as the sha256 checksum of the property `object_type` and properties under `input`, including the start and end braces, separated by `+` (plus) symbols
> 🧪 **example:** `meta/dummy+{"value":"you are now breathing manually"}` -> `5aea ... b0ca`
`full_sha256` is calculated as the sha256 checksum of the property `object_type`, the timestamp `hashed`, and the categories `input`, `local`, and `remote`, including the start and end braces, separated by `+` (plus) symbols
> 🧪 **example:** `meta/dummy+2025-01-01T00:05:00Z+{"value":"you are now breathing manually"}+{}+{}` -> `7f17 ... cbdc`
### logs
> 🚧 **under construction**
>
> **this section needs more expansion!** you can contribute by documenting how this feature or module operates in the latest available version (also check for pull requests that would change that)
| property | transforms | subobjects | conditions | duplicates | value |
| ---------------- | ------------------------ | ---------- | ---------- | ---------- | ------------------------------------------ |
| `logs.level` | `meta/text:value` | none | none | false | can be `error`, `warn`, `info`, or `debug` |
| `logs.variant` | `meta/text:value` | none | none | false | code for the category of the message |
| `logs.timestamp` | `meta/timestamp:unix-ms` | none | none | false | timestamp of the message |
| `logs.message` | `meta/text:value` | none | none | false | human-readable message |
## transforms
a transform is an operation involving taking any property of an object instance, and placing its value into that of a property under `input` of a new object. these are some examples of common transforms:
- existing `meta/text:local.length` -> new `meta/number:value`
- existing `net/url:local.domain` -> new `net/domain:value`
- existing `social/twitter.tweet:remote.posted` -> new `meta/timestamp:unix-ms`
outgoing transforms are defined in the [template of the source object](#transforms-1) under the source property with the destination property, which must always be under `input`.
if an entire object template is used as an outgoing transform, a level of nesting is created within the source property, where the properties under `input` of the destination object template are placed.
similarly, a level of nesting is created if a property's name contains a `.` (dot).
## subobjects
objects can have parent-child relationships with other objects as defined in the parent object's template. this is commonly used when a piece of data is categorized under another in the data source -- these are some examples:
- `social/twitter.tweet` is a subobject of `social/twitter.account`
- `social/discord.message` is a subobject of `social/discord.channel` and `social/discord.thread`
- `web/github.repository` is a subobject of `web/github.user` and `web/github.organization`
subobjects are not object instances nested within each other! it's simply a parent-child relationship between two first-class sovereign objects
similarly to [transforms](#transforms), subobjects are defined in the [template of the parent object](#subobjects-1), along with the shared property (most commonly an identifier). this shared property must have the same [outgoing transforms](#transforms-1) in both the child and parent object.
## templates
**an object template ([`wgu::modules::ModuleItem::Template`](modularity.md#moduleitem-template) is a [toml](https://toml.io) blueprint of an `wgu::ObjectInstance`.** templates are used to configure how object instances behave.
for referencing templates, the name of the module and the name of the template are used, separated by a `/` (slash) symbol.
to refer to a property within a template, the same rules apply as with object instances. add a `:` (colon) symbol, followed by the category (`input` | `local` | `remote`, unless you're referring to a transform destination, in which case it's always going to be `input`), followed by a `.` (dot) and the property name, which can be nested with `.` (dots).
> 🧪 **example:** `meta/text:input.value`
for example, this is the template of the [`meta/dummy`](meta-dummy.md) object type.
```toml
[input.value]
transforms = ["meta/text:value"]
subobjects = []
conditions = []
duplicates = false
```
which then looks like this in the documentation:
| property | transforms | subobjects | conditions | duplicates | value |
| -------- | ----------------- | ---------- | ---------- | ---------- | ---------------- |
| `value` | `meta/text:value` | none | none | false | dummy text value |
### `transforms`
> see also: [transforms](#transforms) in general
array of template types and their properties under `input` that this property transforms to, separated by a `:` (colon) symbol
### `subobjects`
> see also: [subobjects](#subobjects) in general
array of templates and their properties under `input` that this property matches the [outgoing transforms](#transforms-1) of
### `conditions`
must be referring to another property (can be itself). array of values that property must have for this property to be valid in the object instance. **this is mostly there for `input.variant` shenanigans and is NOT to be used for data validation!**
an example would look like this:
```toml
conditions = [
["input.value", "owo"],
["input.variant", "fact-checked by real transgender patriots"]
]
```
### `duplicates`
boolean indicating if this property can be duplicated.
if `true`, it will use an array when represented in json. it can only contain properties specified in `transforms`. when used as transform destinations, duplicable properties add the source property's value to the first (0th) position, and the remaining positions can be filled later by [functions](functions.md).