Separator to avoid bleeding up of the next admonition into the previous image
!!! tip
This is a litle prototype to put the [Malleable Systems Catalog](https://malleable.systems/catalog/) inside the pretty malleable [TiddlyWiki](https://tiddlywiki.com/) (TW),
using [GToolkit](https://gtoolkit.com/) (GT) interative documentation, poweredy by [Pharo](https://pharo.org/).
It was written completely on [Lepiter](https://lepiter.io/feenk/introducing-lepiter--knowledge-management--e2p6apqsz5npq7m4xte0kkywn/)
and is renderized/serialized enterely using the [Markdeep](https://casual-effects.com/markdeep/) light format,
as detailed below,
and published/versioned/stored only with [Fossil](https://fossil-scm.org/).
*See the source code of this page, from your web browser, for a glimpse of the lightness of such workflow*.
The preliminary results can be seen in the [Malleable Systems Wiki](https://mutabit.com/repos.fossil/malleable-systems).
As said in the introduction of the [Malleable Systems Wiki](https://mutabit.com/repos.fossil/malleable-systems):
> this [...] early prototype has two intents:
>
>* It wants to embody the principles of Malleable Systems by porting part of the [Malleable Systems Catalog](https://malleable.systems/catalog/)
> from a (pretty good but) static site generator (Hugo) to a dynamic malleable site,
> which is done also by combining two pretty malleable systems:
> [TiddlyWiki](https://tiddlywiki.com/) and [Glamorous Toolkit](https://gtoolkit.com/).
>
>* Power imbalances in computing go beyond tools themselves and their critical divides
> (user/developer, OS, programming language, IDE, App, Document, etc),
> and reach roots and fruits in gender, etnitity, class and location.
> It is not a coincidence that this started written in English instead of Hindi, Spanish or Quechua
> (nor a reflexion of the interesting things happening in locattions with those languages).
> And, as cultures are supported (or not) by infrastructures,
> this wiki tries to embody a *dialogue of materialities*
> and between peers in the Global North and Global South,
> that accounts for more diverse voices and subjects
> using, connecting, building, learning, teaching and, in general, thinking+feeling+doing
> malleable systems.
>
> Hopefully this will mean that forks, contributions, merges, variants
> will be seeded, gardened and cross pollinated with proper pace and care.
This is the data story of how we approached such migration/prototyping.
**Important**: We are going to mix Pharo code with TiddylyWiki one.
This is because we want to do things outside the browser, JavaScript and NodeJS.
We combine two live coding environments/platforms:
TiddlyWiki for (web) publishing and Pharo/GT for (desktop) computing.
Because we are coming from Pharo/GT coding, most of the code will be expressed in this language.
How to convert some of such code to TW filters and macros, is left as an exercise to the reader
and we will point to some community forum posts where such conversation is happening
(see the ongoing conversation box).
# Prerequisites
To download and "reproduce" this note from the web you will need to have preinstalled:
* [Glamorous Toolkit](https://gtoolkit.com/):
provides the live coding and interactive documentation environment to write, modify and execute this document.
* [GrafoscopioUtils](https://code.tupale.co/Offray/GrafoscopioUtils):
contains utilities to work with Markdeep and other markup languages and tools in our publishing workflow.
* [TiddlyWikiPharo](https://code.tupale.co/Offray/TiddlyWikiPharo/):
utilities to work TiddlyWiki inside Pharo.
* The [PetitParser](https://github.com/moosetechnology/PetitParser) packages for working with YAML. To install it, from a playground run:
# Populating the data
Let's start by reading the Malleable Systems Catalog in YAML:
And then, let's use it to populate a collection of Tiddlers, as JSON files to be droped into our TiddlyWiki:
With this command we can export all `actorTiddlers` as files to be imported in our wiki:
We drop the tiddlers from our temporal file to get them as actors imported into our wiki:
Now let's create the Actors gallery, with the help of combining Mustache templates (ended with `Mus`) and TiddlyWiki ones (just expressed with TW markup in tiddler's text):
Once we have the `actors` tiddler, we can, again exported as a temporal file to be dragged/imported into the wiki
When imported back the previously produce file to our wiki,
it will look like this:
*Part of the actors imported JSON file as rendered inside TiddlyWiki*
Now we need to make something similar to what was done for the Actor also for the Works,
taking into account
And again we export the collected tiddlers as temporal files to be imported in our wiki:
which, when dropped to the wiki produce the imported data:
*Starting imported catalog works from Hugo to TiddlyWiki.*
Now that we have a kind of good starting wiki as an HTML
by extracting and converting data and dropping it as JSON tiddlers to our seed file,
we can use such file to define a proper wiki object:
We can now update the tiddlers by exporting their contents from the raw HTML file
and importing them back into the wiki:
Just for fun, let's see our wiki as a network:
*Network view of the Malleable Systems TiddlyWiki*
And let's export the wiki as STON files to share it in a diff friendly format:
including the individual tiddlers that were created this weekend for the data migration test from
A preliminar view of the tiddler files contained the migrated data from the Mallable Systems site looks like:
In this section we have showcased how to migrate data
from the static site generator Hugo to the dynamic single page aplication TiddlyWiki
so we can create a more malleable version of the malleable systems site.
The intent is also to start a mediated conversation when this different "digital materialities" enter into dialogue,
maybe facilitating other dialgues among people and communities around those different sites, tools and formats.
In this process we showed the code supporting the process,
and different views of it,
from imported tiddlers inside TiddlyWiki,
to networks graphs in Lepiter/Glamorous,
to files in Fossil.
In the next section we are going to see the customized publishing process behind this very document,
that is also part of the wiki.
In such way, the wiki contains not only data and dynamic views of it,
but also "replicable" computational procedures on how it is created.
As with happen with the Hugo to TiddlyWiki migration,
such agile customizations can be incorporated to permanent workflows and/or be the base of futher customizations in other contexts/prototypes.
# Sharing and publishing code stories
The preliminary prototype has been shared in the TiddyWiki and Malleable Systems communities
(see the *Ongoing conversation* box for more links).
!!!
**Ongoing coversation**: These are thes links about where the conversation is happening:
- [Malleable Systems wiki by combining two malleable systems](https://talk.tiddlywiki.org/t/malleable-systems-wiki-by-combining-two-malleable-systems/2122).
- [How to fine tune the migration from Hugo to TW in a TW way?](https://talk.tiddlywiki.org/t/how-to-fine-tune-the-migration-from-hugo-to-tw-in-a-tw-way/2127/3).
- [Malleable Systems Matrix chat channel](https://app.element.io/#/room/#malleable-systems:matrix.org).
- [Twitter short thread with Markdeep an Leptiter authors](https://nitter.snopyta.org/offrayLC/status/1485252352051400706#m).
Next step was being able to share the procudure itself beyond screenshots and deep or ephemeral notices,
depending on where they happened (forums for the first and chat/twitter channels for the later).
For that, I adapted a light format to publish and share [Lepiter](https://lepiter.io/feenk/introducing-lepiter--knowledge-management--e2p6apqsz5npq7m4xte0kkywn/) interactive notes/notebooks on the web,
following the previous explorations on [Grafoscopio](https://mutabit.com/grafoscopio/en.html)
while *combining* [Markdeep](https://casual-effects.com/markdeep/) and [STON](https://github.com/svenvc/ston/blob/master/ston-paper.md).
Previously Grafoscopio documents were stored only in STON
and we had used Markdeep actively in different projects
(see the *Pocket infrastructures powered previous prototypes* box).
By mixing both formats, STON and Markdeep,
the attemp is to keep the idea of a human, diff and [semantic breaks](https://sembr.org/) friendly format
to store and share notebooks of Grafoscopio,
while leveraging on the interactive writing experience of Lepiter
and the publishing smoothness of Markdeep.
!!!
**_Pocket infrastructures_ powered previous prototypes**: These are some
of the projects where we have been exploring the light publishing platforms and workflows
showcased here :
- porting the [Data Journalism handbook](https://mutabit.com/repos.fossil/mapeda/) and the [Data Feminism book](https://mutabit.com/repos.fossil/datafem) .
- publishing our own [Documentathon](https://mutabit.com/repos.fossil/documentaton/) booklet.
- the [Coronapedia](https://mutabit.com/repos.fossil/coronapedia/) minisite.
We collect them under the [pocket infrastructures][pocket] examples umbrella
[GrafoscopioUtils][gfUtils] seemed like the best place to start the previous mentioned integrations and others improvements, migrations and bridges,
as it was small enough and it contained the libraries we used to interact with other tools and formats
(Markdeep, HedgeDoc, Fossil, etc).
So I extended some Lepiter objects (`LeTextualSnippet`, `LeCodeSnippet`, `LePictureSnippet` and `LePage`)
in the GrafoscopioUtils package, to teach them how to export themselves as Markdeep.
How to produce a Markdeep file from a Lepiter note is detailed in the Downloads and updates section.
But you can have a glipmse of three views of the same information in the following picture:
1) inside Lepiter 2) as Markdeep inside the browser and 3) as Markdeep in a text editor.
Observe (by clicking on the image to enlarge) how the editor at the right contains, inside `div` tags, all the medatada required to reconstruct the Lepiter interactive tree document:
The result is what you are reading here.
And despite being an initial prototype with lot of improvements to be done,
it is good enough after almost a week of code work on it,
to share, publish and exchange Lepiter interactive notes
by just sending/publishing a human readable simple and self contained file,
via email, Fossil, Git and/or your favorite web server/folder.
As said, the "extended" Markdeep file contains also all the required metadata to be read back as a nested interactive Lepiter note
and could be the bridge to port years worthy of interactive Grafoscopio documents
(since 2015) to Lepiter notes and notebooks.
# Results, bridges and next steps
The side image shows how the resulting [Malleable Systems Wiki](https://mutabit.com/repos.fossil/malleable-systems) looks like:
![Malleable Systems Wiki Prototype](https://i.imgur.com/SL9XloN.png width="35%")
As said in the wiki, how it evolves would depend on how the conversation evolves.
At the same time, this prototype is an intersection point of longer previous and ongoing explorations,
so other parts would evolve independiently.
This is a list of such ongoing explorations
(checked means almost done, dot means in progress, unchecked means to be started/evaluated):
- [x] A semi-automated procedure to migrate structured data from Hugo to TiddlyWiki,
using the capabilities to create tiddlers and wikis programatically from Pharo
and documeted as Lepiter interactive note.
- [x] A lite format for exporting and *importing* content between Lepiter and Markdeep.
(For more details see the Sharing and publishing code stories section).
- Interactive documentation to version and share individual tiddlers (TiddlyWiki notes)
and improved publishing workflow of wikis, including non versioned files and their backups.
- Better integration of Lepiter, TiddlyWiki, Markdeep, and Fossil in our
[pocket infrastructures](https://mutabit.com/repos.fossil/indieweb/uv/docs/es/miniwiki/grafoscopedia.html#Pocket%20Infrastructures) powered publishing platform,
building on the practices of previous prototypes.
Those improved publication workflows would allow also to replicate / recontextualize,
to certain degree (refer to the Downloads and updates section),
different microsites, including the Malleable Systems Wiki.
A reader/explorer could download an interactive note from the web to Lepiter and
bootstrap the complete lite site/publishing workflow
(that was one of the proposals behind
[Panama Papers: a case for reproducible research, data activism and frictionless data](https://mutabit.com/offray/blog/en/entry/panama-papers-1) ).
- [ ] Support for customized processing of special snippets
(admonitions, footnotes, image properties and so on).
That could be done by using a particular syntax
(like the `%command` on Grafoscopio or the `>>player` in FoxDot)
and/or by specializing the `LeTextualSnippet` objects.
- [ ] Improved preview workflow, maybe by loading some miniserver and/or talking with the web browser.
- [ ] TiddlyWiki alike macros? A great source of user available personalization in TW comes from macros.
Something similar could be done in Markdeep exportation via JavaScript templating engines.
- [ ] Improved outlining in Lepiter: outlining and tree like document should occupy a more prominent place in the writing experience,
similar to OrgMode, Leo Editor or Grafoscopio.
- The use of this computing/publishing platform for participatory civic tech projects. Stay tunned!
# Downloads and updates
!!!
Familiarity with Pharo/GT to load and execute code is presupposed to follow this instructions.
Please refer to introductory tutorials and documentation in both communities/sites to know more about this.
As soon as we have documentation to them in English (we document primary in Spanish) we will point to them here.
Once you have installed the [prerequisites](#prerequisites), to load this document into Lepiter, open a playground and execute:
If the page already exists in the database, you will get an error string message (to be improved),
like the one underlined in the box of the following figure:
Once the Markdeep document is imported in your Glamorous environment,
you can run and adapt it as a Lepiter interactive note.
To export it again as Markdeep, open the note inspector and run:
~~~ Smalltalk
self asMarkdeepFile
~~~
as shown in the following figure:
Notice that the Markdeep file will be stored as a temporal file by default.
If you want to change the exported location of the page,
from the page inspector playground execute:
~~~ Smalltalk
self options at: 'storage' put: FileLocator / 'mySubfolder'
~~~
where `mySubfolder` is a path to the folder where the exported note will be stored.
Regarding updates for this document:
* The last version will be always available at: [markdeepWebPage](https://mutabit.com/repos.fossil/malleable-systems/doc/trunk/wiki/en/malleable-systems-wiki--23fm1.md.html').
* You can verify if you are reading the last version:
* quickly, by checking the semantic version at the top of the page, after the title and author(s) data.
* in detail, by looking at the [page file history](https://mutabit.com/repos.fossil/malleable-systems/finfo?name=wiki/en/malleable-systems-wiki--23fm1.md.html&m=500591006d201ab3).
Regarding updates for the referenced software,
you can check their repositories at the Prerequisites section
and follow the usual update procedure in Pharo/GT systems.
!!! Warning
**Rookie code under modularization and rewriting**.
[Grafoscopio][gf] was the first "serious program" I ever wrote and the one I used to teach myself Pharo,
with the help of the community, but without formal training or companionship
(for several years I was the only active Smalltalker I was aware of in my country,
but with our workshops that's changing slowly).
And you can tell it by looking at the source code of Grafoscopio and several related tools,
like [GrafoscopioUtils][gfUtils].
Now, with more experience,
I'm improving the code while still learning and trying to port some good/"old" ideas from Grafoscopio to Lepiter
(starting with the light human readable serialization formats).
Also, I'm trying to keep compatibility with our pocket infrastructures workflows and explorations
and to bridge our past, since late 2014, when Grafoscopio started,
to the infrastructure that is being provided, since 2021, by Lepiter/GT.
If you review [GrafoscopioUtils][gfUtils], you will see a lot of place to improvement,
particularly on the modularization front, (PR, mentorship, pair sessions are welcomed),
to make a clear separation of what is "core" to
our experience of interacting with external workflows and tools
and what can be loaded as such workflows become more complex.
Grafoscopio may become some kind of plugin bundle to customize the Lepiter experience and build on top of it,
in similar way to what is provided by [Doom Emacs](https://github.com/hlissner/doom-emacs) over "plain" [Emacs](https://www.gnu.org/software/emacs/),
with particular emphasis on data storytellling and visualization for non-coders, civic tech and critical code/data literacy using pocket infrastrucutres and bridging our history of 700+ hours of workshops
(called [Data Weeks](https://mutabit.com/dataweek/) and [Data Rodas](https://mutabit.com/repos.fossil/indieweb/uv/docs/es/miniwiki/grafoscopedia.html#Data%20Rodas)) and our
[several projects and documents and microsites](https://mutabit.com/repos.fossil/offray-blog/uv/bliki/#Portfolio),
powered by Grafoscopio interactive documentation, to what is possible now with Lepiter/GT.
Time will tell. Expect some changes.
# Conclusions
As we did before with Grafoscopio, but now with the facilties of Lepiter,
we use interactive documentation to approach a problem/prototype by bridging several tools in a customized agile workflow,
while extending part of such tools, in this case:
Lepiter, TiddlyWikiPharo, GrafoscopioUtils (all them via custom code), TiddlyWiki (via macros) and Markdeep (via STON metadata divs).
Our agile prototypes/workflow included:
* A way to migrate structured information in light formats (YAML) from a static site generator (Hugo) to a dynamic malleable wiki (TiddlyWiki).
* An improved workflow to serialize and publish Lepiter interactive notes using Markdeep, STON and Fossil.
* Talks with the local and international communities and authors behind several tools we combine here.
* And will incorporate, as usual, critical code/data literacy and civic tech workshops
with the local grassroots communities to share and expand on this learning, tools and workflows in other contexts.
Regarding the talks, as medium influences communication,
the ones in the TiddlyWiki Forum, have the reflexive and detailed tone that this place allows,
while the ones in Matrix, Discord and Twitter have the more ephemeral and discontinous nature of such places.
I have enabled another place of conversation here,
via the Hypothesis powered comment of this page.
And I hope to have more and longer public talks about this workflows with Markdeep and GT/Pharo communities/authors in a forum like style in addition to the ephemeral talk/chat like style.
Maybe using the GitHub discussions instead of chat channels and twitter."
Our approach showcased a pretty good simplicity versus expressivity ratio,
that is embedded in our criteria to choose and combine *pocket infrastructures*
and it bridging our previous history, learnig and practices
to what is possible recently as result of another decade long (re)search.
> "Hopefully this fluent process will be an intersection point of many people, tools and communities,
> (TiddlyWiki, Pharo, Glamorous Toolkit, Malleable Systems, Grafoscopio and Markdeep)
> and a small showcase of what is possible when you bridge them
>and the kind fruits of a longer conversation ahead
> with deep roots in the storylines of each community and person with mutable technologies."
# Licenses & Collaborations
Licenses are a statement about what we care about (and what we don't).
So, the software and its interactive documentation is covered by pretty liberal but different licences,
allowing the related works keep contributing to the digital commons, the communities around them and their ecosystems,
while acknowledging the complementary and different nature of writing and publishing software and documentation,
even when interactive documents blur the distinction.
In a similar way to [Pharo Books](https://books.pharo.org/) and other documentation efforts covering free/libre software:
* the source code of the software
(Pharo, GT, TiddlyWiki, TiddlyWikiPharo, GrafoscopioUtils, Markdeep, Fossil)
is convered under MIT or BSD licenses (see each one's license for details),
* the documentation, in particular this one,
is covered under a Creative Commons *inspired* variant that allows free distribution and modification for non commercial pursposes
with an exception in our case:
if you're an individual, a solo merchant, member of a workers owned cooperative or owner
of a small/medium business, you can also use/mofify this documentation for commercial purposes,
as long as you keep the original license over derivated works.
You can see details about such license in the
[P2P+ License](https://mutabit.com/repos.fossil/malleable-systems/uv/wiki/#P2P%2B) (in Spanish).
Hopefully this will encourage individuals, cooperatives and small/medium business
to put time and make profit of this documentation,
and also invite bigger business to invest in the digital commons, its documentation and people,
and give back if they plan to use such commons in some way.
If you want to contribute, have inquiries about this document or the licensing
want to sublicense it or have any other question regading this efforts,
please contact us at:
