**Grafoscopio Readme**
This `readme` file introduce [Grafoscopio][grafoscopio-en] a tool for interactive documentation,
reproducible research and publishing, in a quick way, to have a panoramic
view of it and to make it work easily.
It is a edited shorter version of the [full user manual][full-manual], please refer to
it for detailed documentation.
# Overview
![Figure [fig:web-page-en]: Detail for the Grafoscopio [English web page](https://is.gd/grafoscopio_e).](./Docs/Imagenes/grafoscopio-webpage-en.png)
## Quick Start
Grafoscopio works on [Pharo][pharo] 8.x .
If you already know Pharo, just open a [playground][play] copy the following script and execute
it:
~~~ Smalltalk linenumbers
location := FileLocator localDirectory / 'iceberg' / 'Offray' / 'Grafoscopio'.
(IceRepositoryCreator new
location: location;
remote: (IceGitRemote url: 'https://code.tupale.co/Offray/Grafoscopio.git');
createRepository)
register.
Metacello new
repository: 'gitlocal://', location fullName;
baseline: 'Grafoscopio';
load
~~~
[Grafoscopio installation script.]
After that you can execute several short scripts to browse Grafoscopio notebooks.
See the Examples section for more details.
If you want a to know more about Grafoscopio and the installation process, continue reading.
## This tool and you
Grafoscopio is a moldable tool for literate computing and reproducible
research developed on [Pharo](http://pharo.org/) live coding and computing integrated
environment, in the context of a PhD research in a hacker space of the Global South
([HackBo](http://hackbo.co/) in Bogotá, Colombia), that is being actively used, developed and documented.
We will expand on the points of the previous definition.
- Being moldable [@moldable-debugger-2014] means that is easy to adapt the tool to
several problems, which follows the opposite popular path of trying to force
several problems into a predefined tool.
Tools change us. So we need to change them back to express our concerns and to
help us in dealing with complex issues and problems in a flexible way.
- Literate computing [@literate-computing-2015] is a way of intertwining prose,
code, data and visualizations to make data based storytelling, experimentation,
exploration and documentation in diverse broad fields like academia, journalism,
research and (h)ac(k)tivism.
Grafoscopio allows the creation of literate computing structured interactive
notebooks, that take the form of a tree-like programmable document.
- Research claims and findings and supporting data and visualizations can be
integrated in interactive notebooks with historic traceability, allowing
reproducible research.
- Because of the continuity and uniformity of the Pharo environment [@pbe-2016],
Grafoscopio blurs the distinction between code, data, document, application and
IDE [^ide] and tries to blur also the distinction between interactive documents
authors and software developers.
- From the particular context where is being developed (HackBo hackerspace and a
PhD research on Design and Creation), Grafoscopio is concived as a empowering
simple and self contained *pocket infrastructure* (that work off-line/on-line
from USB thumb drives and/or low resources machines [@luna-grafoscopio-2014]),
wich states a critical approach to exclusionary ideas about data, coding,
research, and their practicioners, places, and supporting tools and
infrastructures.
In the [Grafoscopio web page](https://is.gd/grafoscopio_e), we showcase several
projects aligned with such critical approach and the idea that technology is
mostly not neutral, wich is also reflected in some parts of this text and in the
fact that we're opinionated about technology and its uses.
I hope you find our technology and themes choices empowering and reavealing of
alternatives.
Grafoscopio is intended to be used by learners and researchers in several fields:
academia, journalism, activism, hacktivism and for anyone interested in open
reproducible research and data storytelling backed by data and agile visualizations
[@bergel_agile_2016].
This document assumes that you are such person.
We will introduce the general features of Grafoscopio and point to several external
and internal resources to complete your panoramic view of the ecosystem that let you
deep your knowledge.
We included introductory resources to learn Pharo and data visualization,
processing and storage related technologies (see the `Help` menu), and
the [Grafoscopio web page](https://is.gd/grafoscopio_e) (see figure
[fig:web-page-en]) shows several examples of how to use them for specific projects:
Panama Papers as reproducible research; Twitter data selfies; specific domain visualizations for medicine information; open, garage and citizen science and
research.
*This whole manual was also created using Grafoscopio* and is also an example of the
type of documents you can create with this tool.
We hope to inspire you to create and publish your own projects.
This document, by not trying to be comprenhensive, is a first invitation to know the
Grafoscopio environment and to deep your knowledge with the use of it and other
related resources.
You will see that, despite of being a manual, it includes pretty practical examples
and invitations.
That is because I think that learning something new is more like reading a map
that reading a manual: you make first a panoramic view of where you are and
where you want to be, and take a practical approach to making your tasks and
reaching your destination.
_No prior knowledge of programming is supposed to follow this manual_.
!!! note *A prototype pointing to future possibilites*
Despite of being pretty usable, you will see that Grafoscopio is not totally finished,
and this shows in a few spots of the Graphical User Interface (GUI) that "point to the
future", towards funtionality still to be implemented.
It's an unusual approach, but I think that is important to convey some sense of possibility,
and work to be done in the GUI, instead of a fully polished "product" or a GUI that hides
what is not ready.
This conviction comes from the [hackathons and workshops](https://mutabit.com/dataweek/)
where we worked and evolved Grafoscopio, while the workshop was happening(!), thanks to the
dynamic, moldable and continuous nature of the Pharo live coding environment.
Blurring the distinction between interactive documents authors and software developers,
means to put the whole environment at their dispossal, and to show the community that they
can be part of this future possibilities, and that is why we take this unusual approach to
GUI.
Where the GUI is more a remainder for the future, I will point that using the **TBD**
remark (for To Be Done).
## Place in the ecosystem
[top][top]
### Similar tools
Grafoscopio is similar to other tools and has been inspired by many
of them, while is trying to bring also new possibilities, by combining
different ideas, diverging from others, puting "parallel" ideas into
dialog and, hopefully, bringing new ones.
Here we talk about the similarities and differences with other tools.
- Like [Jupyter][jupyter], or [Zeppling][zeppling], [Beaker][beaker]
or [nteract][nteract], Grafoscopio provides interactive notebook functionality,
but it is focused only on Pharo code right now, instead of being a
"language neutral" notebook (but this could be a feature for the future).
Grafoscopio is a multiplatform (Mac, Windows, Gnu/Linux) desktop
application (like nteract, or Electron Beaker), instead of a web one
(like Jupyter, Zepelling or Beaker), providing a simple, extensible,
powerful, self-contained and portable environment for interactive computing,
(as said it can run from a USB thumb drive, modest computers
and anything in between and beyond).
- Grafoscopio organizes documents in a tree like metaphor, also called the *outline*, or the
notebook, that is interactive and programmable, like [Org Mode][org-mode],
[Leo Editor][leo], [TeXmacs][texmacs] or [Pollen][pollen] and share with them the idea that
the "document is the program"[^book-program] (or a programable container of small chunks of
programs and scripts).
Also, the document author, can define custom tags that can be used to traverse the
document tree and produce multiple customized views and outputs from a single document.
A single notebook can contain short or book size interactive documents
(this full manual is in a single Grafoscopio notebook).
- Like [Jupyter Lab][jupyterlab], Grafoscopio environment supports needs beyond the notebook.
Grafoscopio achieves this by leveraging the advantange of the extensible Pharo computing
environment and ecosystem, where it inhabits, with powerful tools for several computing
taks, beyond and complementary to interactive documentation and reproducible research: GUI
bulding, data processing and visualization, unit testing, code repositories and source
management, among others.
- Grafoscopio uses the [Roassal agile visualization engine][roassal], to build interactive
visualizations and export them to the web.
Roassal provides similar functionality to other visualization engines and toolkits like
[d3.js][d3js], [RaphaelJS][raphaeljs], [Processing][processing] or [Flare][flare], but, by
being part of the Pharo live coding environment, it invites to a more explorative and
dynamic building of visualizations in an agile way.
- At the moment, notebook sharing and collaboration and print (PDF) and web (HTML) publishing
are supported, but in the future we hope to provide advanced interactive notebook publishing
features in a distributed p2p fashion (see next section for the techologies that enable
this).
### Technologies behind
Grafoscopio tries to become a simple, understandable, moldable,
versatile and flexible tool thanks to the power of [Pharo][pharo]
environment and ecosystem and the combination with mature external
and internal frameworks and tools. It uses:
- Internal tools and frameworks:
- [Spec][spec] for embeddable code playgrounds, GUI and interactive notebook nodes.
- [Roassal][roassal] for data visualization.
- [STON][ston] for a light data storage and a human friendly notebooks format.
- [NeoJSON][neojson] for interacting with structured hierarchical [JSON][json] data.
- [Zinc Http Components][zn]: To interact with the web.
- External tools and frameworks:
- [Fossil SCM][fossil] for collaboration, publication and traceability of the documents
history (including this very document).
- [Git][git]/[Gitea][gitea]: For software source code management.
- [Pandoc][pandoc] for exporting to printing (PDF) and web (HTML) formats.
- [SQLite][sqlite] for storage and management of tabular data, for the `Dataviz` companion
package.
[gitea]: https://gitea.io/
Despite of trying to provide a friendly, cohesive and empowering user experience (UX) by
integrating default external minimalist and/or self-contained tools into the data exploration
and document publishing workflow, other external tools could be integrated (more data bases,
including [NoSQL][nosql], other exporters and [light markup languages][light-markup-languages]
and so on).
# Installation instructions
Installing Grafoscopio on Pharo 8.x (tested with 8.0.0) is just a matter of running a
script from the [playground][play].
There are two ways of running scripts in the Pharo environment: one by importing them
from the Internet and the other by writing them manually.
We will see them in detail now.
## Run a playground from Internet
If you want to run a Pharo script from internet, open the spotter
and paste the script address and then press `Enter` to open
the interactive *playground* and finally press the `Do it and go` green play
button or its shorcut (`Ctrl + Shift + g`).
To install Grafoscopio vía an Internet script
1. Open the spotter (`Shift + Enter`) and paste this script address:
http://ws.stfx.eu/84I7YYWMJNZ6.
You will see a screenshot similar to figure [fig:install-script-part1],
showing the web address you have pasted and the first lines of the script
below, marked in grey.
![Figure [fig:install-script-part1]: Loading the install configuration package. ](../../uv/Docs/Imagenes/playground-from-spotter.png)
2. Press `Enter` or select with the mouse the area with the grey background.
You will see the interactive [playground][play] with the script loaded, as shown here:
3. Press the play button or the shorcut (`Ctrl + Shift + g`).
You will see that the playground is being executed the installation progress bars.
4. Once the installation playground has been executed it will contain a new column with
details of the object resulting from that execution.
Also you will see a new menu entry saying "Grafoscopio" in the top bar, at the right of the
"Pharo" entry in the top left corner.
The result is shown in figure [fig:executed-playground].
![Figure [fig:executed-playground]: Grafoscopio installed.](../../uv/Docs/Imagenes/Install/instalacion-instalado.png)
[play]: https://github.com/pharo-open-documentation/pharo-wiki/blob/master/General/Playground.md
## Run a playground with a script
You can also open a new playground (`Ctrl` + `o` + `w`), you will see something
like the figure [fig:empty-playground])
![Figure [fig:empty-playground]: Empty *playground* and its play button. ](./Docs/Imagenes/Install/empty-playground-1.png)
and write or copy and paste the script presented in the
Quick Start and finally execute the playground, pressing the
`Play button`.
The installation procedure will show progress bars and a resulting main menu, as
described before.
## Save the installation in your Pharo computing environment
!!! Tip: **Important**
Once we have Grafoscopio installed, by any of the means shown in this section, you need to
to save the modifications to our computing environment, by making click in the `Pharo`
entre of the main menu or in any clean part of the GUI (not occupied by any window)
to deploy the *World Menu*.
One there, choose `Save`, to save the system with the same name, or `Save as` to save
it with a new one (see figure [fig:install-screen5]).
![ Figure [fig:install-screen5]: Saving changes to our Pharo environment via the World Menu. ](./Docs/Imagenes/Install/save-menu.png)
# Usage instructions
[top][top]
For detailed usage instruction please read the extended version of this `readme`
file in the [Grafoscopio User Manual](./Docs/En/Books/Manual/manual.pdf).
This short `readme`, derived from that manual, follows the [JOSS][joss] outline for
a quick overview of the software.
# Examples
There is a dedicated complementary package, called `Dataviz`, with examples,
that was build with educative purposes, for a recurrent local workshop called
the Data Week, where we invite a diverse community in gender, professions,
educational and life backgrounds and is automatically installed when you
install Grafoscopio.
Also we have a Spanish introductory tutorial, that is directed towards beginners.
To see such examples please execute this code form the *playground*:
~~~
"This opens the Spanish tutorial"
GrafoscopioNotebook open: GrafoscopioDocs tutorial
~~~
After installing the [Dataviz package][dataviz] you can run the following
script to see several domain specific applications:
~~~
"This opens the introductory notebook to the Dataviz package"
GrafoscopioNotebook open: DatavizDocs introNotebook
~~~
# API documentation
Because Grafoscopio inhabits a full live coding environment,
it follows the custom of making the API documentation available
inside a dynamic environment, instead in some static web page.
To open the Grafoscopio package inside the system browser,
and see the messages organized by protocols execute:
~~~
"Browser the notebook API"
GrafoscopioNotebook browse.
"Browse the document tree API"
GrafoscopioNode browse.
~~~
# Tests
The core functionality of Grafoscopio document tree model is tested.
If you want to see and run the tests, just open the tests package by
executing:
~~~
GrafoscopioNodeTest browse
~~~
From there, you can use the extensive test environment provided by Pharo.
For a short introduction to how to run test on this environment,
I recommend to watch the [Coding a Little Domain Specific Language video ][mooc-w3-14],
from the [Pharo MOOC][mooc].
# Known bugs
There are not known critical bugs now.
But the non saved documents don't show any advice if you close them.
This is a big usability issue and as we migrate to the new graphical
interface, called Spec2, we hope to improve greatly on usability.
!!! WARNING
Double check you have saved your notebooks before closing them.
We are going to hunt and squeeze that bug out of existance.
Resistance is futile.
To help us with this or other bugs please look at the Community Guidelines to
know how to contribute to the project.
# Community Guidelines
## Seek support
Grafoscopio has a small and new born community.
You can reach it by following the contact links
in the Grafoscopio page in
[Spanish](http://mutabit.com/grafoscopio/)
or in [English](http://mutabit.com/grafoscopio/index.en.html).
Also you can discuss issues related with Grafoscopio in the
[Pharo users community](http://pharo.org/community)
mailing list.
We are in such list and try to be active participants there
and bridge the local Spanish community with the international
one.
## Report issues or problems
[top][top]
To report issues or problems with the software and/or its documentation
please visit our [ticket section][grafoscopio-tickets] Fossil repository.
Before creating a new ticket, please be sure to visit the
[current tickets][grafoscopio-tickets-current], to see if your issue/problem has been
not reported already.
## Contribute to the project
As we said, Grafoscopio wants to help in blurring the distinction
between software developer and interactive document author,
so we are pretty open to several ways of contribution: from
bug reports, as explained above, to the creation of interactive
documentation, domain specific languages (DSLs) and visualizations,
or software functionality.
Contributions usually take part on our recurrent [Data Week][dataweek]
hackathon/workshop and there you will learn how to use and adapt
the software, starting by the basics, creating DSLs and crafting
visualizations and integrating them into interactive notebooks.
You will also learn how to use Fossil and how to commit to our
shared repositories for [code][grafoscopio-sthub] and for
[documents and issues][grafoscopio-fossil].
Besides this manual, we are creating also a tutorial (in Spanish) with all
these themes covered, as memories for us and others to remember and learn from.
The idea, as was said before, is to have multilingual documentation with a *local
first* approach.
If you don't have the chance to assist to one of our face to face learning workshops
and hackathons or to use the resulting notebooks, but still want and already know
who to contribute, you can also ask for permisions in the respositories using any of
the contact methods listed above.
We are a small, new born and friendly community with low traffic
mail communication and can discuss about contributions on an
individual case by case approach, so your words, bugfix and suggestions
will be listened and taking into account and integrated when and where they
make sense.
Welcome again to our community :-).
# Licenses
Grafoscopio and its tutorial is licensed under MIT license and the readme and the User
Manual, and other documentaiton is licensed under a modified P2P license.
To see a full copy of such respective licenses, please visit the files under this repository:
- [Grafoscopio MIT License](./Docs/En/Licenses/grafoscopio-mit.md)
- [Documentation P2P License](./Docs/En/Licenses/documents-p2p-ismb.md)
[^ide]: IDE: Integrated software Development Environment
[^book-program]: The idea of the "document is a program" is a paraphrasis of "the book is a program",
stated in the Pollen documentation, which is a short phrase to express a powerful idea
about burring the disctinction between the document and the program, that is present
in several programs, like TeXmacs, Leo, Org Mode, and, of course, Grafoscopio.
[agileviz-quickstart]: https://dl.dropboxusercontent.com/u/31543901/AgileVisualization/QuickStart/0101-QuickStart.html
[beaker]: http://beakernotebook.com/
[bibtex]: https://en.wikipedia.org/wiki/BibTeX
[chocolatey]: https://chocolatey.org/
[citezen]: http://people.untyped.org/damien.pollet/software/citezen/
[d3js]: https://d3js.org/
[dataweek]: http://mutabit.com/dataweek/
[dataviz]: https://mutabit.com/repos.fossil/dataviz/
[etherpad]: https://en.wikipedia.org/wiki/Etherpad
[flare]: http://flare.prefuse.org/
[floor-function]: https://en.wikipedia.org/wiki/Floor_and_ceiling_functions
[fossil]: http://fossil-scm.org/
[fuel]: http://rmod.inria.fr/web/software/Fuel
[full-manual]: ../../uv/Docs/En/Books/Manual/manual.pdf
[joss]: http://joss.theoj.org/
[grafoscopio-en]: http://mutabit.com/grafoscopio/index.en.html
[grafoscopio-fossil]: http://mutabit.com/repos.fossil/grafoscopio/
[grafoscopio-tickets]: http://mutabit.com/repos.fossil/grafoscopio/ticket
[grafoscopio-tickets-current]: http://mutabit.com/repos.fossil/grafoscopio/rptview?rn=1
[grafoscopio-sthub]: http://smalltalkhub.com/#!/~Offray/Grafoscopio
[git]: https://en.wikipedia.org/wiki/Git
[github]: https://en.wikipedia.org/wiki/GitHub
[gogs]: https://gogs.io/
[gt-tools]: http://gtoolkit.org/
[hackbo]: http://hackbo.co/
[json]: http://en.wikipedia.org/wiki/JavaScript_Object_Notation
[jupyter]: http://jupyter.org/
[jupyterlab]: http://jupyterlab.github.io/jupyterlab/
[latex]: https://en.wikipedia.org/wiki/LaTeX
[leo]: http://leoeditor.com
[light-markup-languages]: https://en.wikipedia.org/wiki/Light_markup_language
[manual-tip]: http://mutabit.com/repos.fossil/grafoscopio/dir?tip&name=Docs/En/Books/Manual
[markdown]: http://pandoc.org/MANUAL.html#pandocs-markdown
[mooc]: https://siemenbaader.github.io/PharoMooc-website/
[mooc-spotter1]: http://rmod-pharo-mooc.lille.inria.fr/MOOC/Videos/W3/C019-Videos-Spotter1-BrowsingAClassWithSpotter-V2-HD_720p_4Mbs.m4v
[mooc-spotter2]: http://rmod-pharo-mooc.lille.inria.fr/MOOC/Videos/W3/C019-Videos-Spotter2-Categories-V2-HD_720p_4Mbs.m4v
[mooc-w3-14]: http://amara.org/en/videos/04UWGMwnrdXz/info/rmod-pharo-mooclilleinriafrc019-videos-redo-dsl-hd_720p_4mbsm4v/
[neojson]: https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/NeoJSON/NeoJSON.html
[nix]: http://nixos.org/nix/
[nosql]: https://en.wikipedia.org/wiki/NoSQL
[nteract]: https://nteract.io/
[oop-pharo-2017]: https://ci.inria.fr/pharo-contribution/view/Books/job/LearningObjectOrientedProgramming/98/artifact/book.pdf
[org-mode]: http://orgmode.org/
[pastebin]: https://en.wikipedia.org/wiki/Pastebin
[pandoc]: http://pandoc.org/
[pharo]: http://pharo.org/
[pharo-download]: http://pharo.org/download
[pollen]: http://docs.racket-lang.org/pollen/
[processing]: https://processing.org/
[raphaeljs]: https://dmitrybaranovskiy.github.io/raphael/
[roassal]: http://agilevisualization.com/
[rolling-release]: https://en.wikipedia.org/wiki/Rolling_release
[spec]: http://files.pharo.org/books/spec-tutorial/
[spotter]: http://gtoolkit.org/#spotter
[sqlite]: http://sqlite.org/
[ston]: https://github.com/svenvc/ston/blob/master/ston-paper.md
[texmacs]: http://texmacs.org/
[top]: #
[udbc]: http://www.smalltalkhub.com/#!/~TorstenBergmann/UDBC
[zeppling]: http://zeppelin-project.org/
[zotero]: https://www.zotero.org/
[zotero-manual]: https://www.zotero.org/groups/diseo_y_creacion_phd_msc_universidad_de_caldas/items/collectionKey/PHGTCZVT
[zn]: http://zn.stfx.eu/zn/index.html