**Grafoscopio Readme**
Home | Timeline | Tickets | Files | RSS
This `readme` file introduce [Grafoscopio][grafoscopio-en] a tool for interactive documentation, reproducible research and publishing, in a quick way, to have a panoramic view of it and to make it work easily. It is a edited shorter version of the [full user manual][full-manual], please refer to it for detailed documentation.
# Overview ![Figure [fig:web-page-en]: Detail for the Grafoscopio [English web page](https://is.gd/grafoscopio_e).](./Docs/Imagenes/grafoscopio-webpage-en.png) ## Quick Start Grafoscopio works on [Pharo][pharo] 8.x . If you already know Pharo, just open a [playground][play] copy the following script and execute it: ~~~ Smalltalk linenumbers location := FileLocator localDirectory / 'iceberg' / 'Offray' / 'Grafoscopio'. (IceRepositoryCreator new location: location; remote: (IceGitRemote url: 'https://code.tupale.co/Offray/Grafoscopio.git'); createRepository) register. Metacello new repository: 'gitlocal://', location fullName; baseline: 'Grafoscopio'; load ~~~ [Grafoscopio installation script.] After that you can execute several short scripts to browse Grafoscopio notebooks. See the Examples section for more details. If you want a to know more about Grafoscopio and the installation process, continue reading. ## This tool and you Grafoscopio is a moldable tool for literate computing and reproducible research developed on [Pharo](http://pharo.org/) live coding and computing integrated environment, in the context of a PhD research in a hacker space of the Global South ([HackBo](http://hackbo.co/) in Bogotá, Colombia), that is being actively used, developed and documented. We will expand on the points of the previous definition. - Being moldable [@moldable-debugger-2014] means that is easy to adapt the tool to several problems, which follows the opposite popular path of trying to force several problems into a predefined tool. Tools change us. So we need to change them back to express our concerns and to help us in dealing with complex issues and problems in a flexible way. - Literate computing [@literate-computing-2015] is a way of intertwining prose, code, data and visualizations to make data based storytelling, experimentation, exploration and documentation in diverse broad fields like academia, journalism, research and (h)ac(k)tivism. Grafoscopio allows the creation of literate computing structured interactive notebooks, that take the form of a tree-like programmable document. - Research claims and findings and supporting data and visualizations can be integrated in interactive notebooks with historic traceability, allowing reproducible research. - Because of the continuity and uniformity of the Pharo environment [@pbe-2016], Grafoscopio blurs the distinction between code, data, document, application and IDE [^ide] and tries to blur also the distinction between interactive documents authors and software developers. - From the particular context where is being developed (HackBo hackerspace and a PhD research on Design and Creation), Grafoscopio is concived as a empowering simple and self contained *pocket infrastructure* (that work off-line/on-line from USB thumb drives and/or low resources machines [@luna-grafoscopio-2014]), wich states a critical approach to exclusionary ideas about data, coding, research, and their practicioners, places, and supporting tools and infrastructures. In the [Grafoscopio web page](https://is.gd/grafoscopio_e), we showcase several projects aligned with such critical approach and the idea that technology is mostly not neutral, wich is also reflected in some parts of this text and in the fact that we're opinionated about technology and its uses. I hope you find our technology and themes choices empowering and reavealing of alternatives. Grafoscopio is intended to be used by learners and researchers in several fields: academia, journalism, activism, hacktivism and for anyone interested in open reproducible research and data storytelling backed by data and agile visualizations [@bergel_agile_2016]. This document assumes that you are such person. We will introduce the general features of Grafoscopio and point to several external and internal resources to complete your panoramic view of the ecosystem that let you deep your knowledge. We included introductory resources to learn Pharo and data visualization, processing and storage related technologies (see the `Help` menu), and the [Grafoscopio web page](https://is.gd/grafoscopio_e) (see figure [fig:web-page-en]) shows several examples of how to use them for specific projects: Panama Papers as reproducible research; Twitter data selfies; specific domain visualizations for medicine information; open, garage and citizen science and research. *This whole manual was also created using Grafoscopio* and is also an example of the type of documents you can create with this tool. We hope to inspire you to create and publish your own projects. This document, by not trying to be comprenhensive, is a first invitation to know the Grafoscopio environment and to deep your knowledge with the use of it and other related resources. You will see that, despite of being a manual, it includes pretty practical examples and invitations. That is because I think that learning something new is more like reading a map that reading a manual: you make first a panoramic view of where you are and where you want to be, and take a practical approach to making your tasks and reaching your destination. _No prior knowledge of programming is supposed to follow this manual_. !!! note *A prototype pointing to future possibilites* Despite of being pretty usable, you will see that Grafoscopio is not totally finished, and this shows in a few spots of the Graphical User Interface (GUI) that "point to the future", towards funtionality still to be implemented. It's an unusual approach, but I think that is important to convey some sense of possibility, and work to be done in the GUI, instead of a fully polished "product" or a GUI that hides what is not ready. This conviction comes from the [hackathons and workshops](https://mutabit.com/dataweek/) where we worked and evolved Grafoscopio, while the workshop was happening(!), thanks to the dynamic, moldable and continuous nature of the Pharo live coding environment. Blurring the distinction between interactive documents authors and software developers, means to put the whole environment at their dispossal, and to show the community that they can be part of this future possibilities, and that is why we take this unusual approach to GUI. Where the GUI is more a remainder for the future, I will point that using the **TBD** remark (for To Be Done). ## Place in the ecosystem [top][top] ### Similar tools Grafoscopio is similar to other tools and has been inspired by many of them, while is trying to bring also new possibilities, by combining different ideas, diverging from others, puting "parallel" ideas into dialog and, hopefully, bringing new ones. Here we talk about the similarities and differences with other tools. - Like [Jupyter][jupyter], or [Zeppling][zeppling], [Beaker][beaker] or [nteract][nteract], Grafoscopio provides interactive notebook functionality, but it is focused only on Pharo code right now, instead of being a "language neutral" notebook (but this could be a feature for the future). Grafoscopio is a multiplatform (Mac, Windows, Gnu/Linux) desktop application (like nteract, or Electron Beaker), instead of a web one (like Jupyter, Zepelling or Beaker), providing a simple, extensible, powerful, self-contained and portable environment for interactive computing, (as said it can run from a USB thumb drive, modest computers and anything in between and beyond). - Grafoscopio organizes documents in a tree like metaphor, also called the *outline*, or the notebook, that is interactive and programmable, like [Org Mode][org-mode], [Leo Editor][leo], [TeXmacs][texmacs] or [Pollen][pollen] and share with them the idea that the "document is the program"[^book-program] (or a programable container of small chunks of programs and scripts). Also, the document author, can define custom tags that can be used to traverse the document tree and produce multiple customized views and outputs from a single document. A single notebook can contain short or book size interactive documents (this full manual is in a single Grafoscopio notebook). - Like [Jupyter Lab][jupyterlab], Grafoscopio environment supports needs beyond the notebook. Grafoscopio achieves this by leveraging the advantange of the extensible Pharo computing environment and ecosystem, where it inhabits, with powerful tools for several computing taks, beyond and complementary to interactive documentation and reproducible research: GUI bulding, data processing and visualization, unit testing, code repositories and source management, among others. - Grafoscopio uses the [Roassal agile visualization engine][roassal], to build interactive visualizations and export them to the web. Roassal provides similar functionality to other visualization engines and toolkits like [d3.js][d3js], [RaphaelJS][raphaeljs], [Processing][processing] or [Flare][flare], but, by being part of the Pharo live coding environment, it invites to a more explorative and dynamic building of visualizations in an agile way. - At the moment, notebook sharing and collaboration and print (PDF) and web (HTML) publishing are supported, but in the future we hope to provide advanced interactive notebook publishing features in a distributed p2p fashion (see next section for the techologies that enable this). ### Technologies behind Grafoscopio tries to become a simple, understandable, moldable, versatile and flexible tool thanks to the power of [Pharo][pharo] environment and ecosystem and the combination with mature external and internal frameworks and tools. It uses: - Internal tools and frameworks: - [Spec][spec] for embeddable code playgrounds, GUI and interactive notebook nodes. - [Roassal][roassal] for data visualization. - [STON][ston] for a light data storage and a human friendly notebooks format. - [NeoJSON][neojson] for interacting with structured hierarchical [JSON][json] data. - [Zinc Http Components][zn]: To interact with the web. - External tools and frameworks: - [Fossil SCM][fossil] for collaboration, publication and traceability of the documents history (including this very document). - [Git][git]/[Gitea][gitea]: For software source code management. - [Pandoc][pandoc] for exporting to printing (PDF) and web (HTML) formats. - [SQLite][sqlite] for storage and management of tabular data, for the `Dataviz` companion package. [gitea]: https://gitea.io/ Despite of trying to provide a friendly, cohesive and empowering user experience (UX) by integrating default external minimalist and/or self-contained tools into the data exploration and document publishing workflow, other external tools could be integrated (more data bases, including [NoSQL][nosql], other exporters and [light markup languages][light-markup-languages] and so on). # Installation instructions Installing Grafoscopio on Pharo 8.x (tested with 8.0.0) is just a matter of running a script from the [playground][play]. There are two ways of running scripts in the Pharo environment: one by importing them from the Internet and the other by writing them manually. We will see them in detail now. ## Run a playground from Internet If you want to run a Pharo script from internet, open the spotter and paste the script address and then press `Enter` to open the interactive *playground* and finally press the `Do it and go` green play button or its shorcut (`Ctrl + Shift + g`). To install Grafoscopio vía an Internet script 1. Open the spotter (`Shift + Enter`) and paste this script address: http://ws.stfx.eu/84I7YYWMJNZ6. You will see a screenshot similar to figure [fig:install-script-part1], showing the web address you have pasted and the first lines of the script below, marked in grey. ![Figure [fig:install-script-part1]: Loading the install configuration package. ](../../uv/Docs/Imagenes/playground-from-spotter.png) 2. Press `Enter` or select with the mouse the area with the grey background. You will see the interactive [playground][play] with the script loaded, as shown here: ![Installation script](../../uv/Docs/Imagenes/installation-script.png) 3. Press the play button or the shorcut (`Ctrl + Shift + g`). You will see that the playground is being executed the installation progress bars. ![Installation progress bars](./Docs/Imagenes/Install/progress-bar.png) 4. Once the installation playground has been executed it will contain a new column with details of the object resulting from that execution. Also you will see a new menu entry saying "Grafoscopio" in the top bar, at the right of the "Pharo" entry in the top left corner. The result is shown in figure [fig:executed-playground]. ![Figure [fig:executed-playground]: Grafoscopio installed.](../../uv/Docs/Imagenes/Install/instalacion-instalado.png) [play]: https://github.com/pharo-open-documentation/pharo-wiki/blob/master/General/Playground.md ## Run a playground with a script You can also open a new playground (`Ctrl` + `o` + `w`), you will see something like the figure [fig:empty-playground]) ![Figure [fig:empty-playground]: Empty *playground* and its play button. ](./Docs/Imagenes/Install/empty-playground-1.png) and write or copy and paste the script presented in the Quick Start and finally execute the playground, pressing the `Play button`. The installation procedure will show progress bars and a resulting main menu, as described before. ## Save the installation in your Pharo computing environment !!! Tip: **Important** Once we have Grafoscopio installed, by any of the means shown in this section, you need to to save the modifications to our computing environment, by making click in the `Pharo` entre of the main menu or in any clean part of the GUI (not occupied by any window) to deploy the *World Menu*. One there, choose `Save`, to save the system with the same name, or `Save as` to save it with a new one (see figure [fig:install-screen5]). ![ Figure [fig:install-screen5]: Saving changes to our Pharo environment via the World Menu. ](./Docs/Imagenes/Install/save-menu.png) # Usage instructions [top][top] For detailed usage instruction please read the extended version of this `readme` file in the [Grafoscopio User Manual](./Docs/En/Books/Manual/manual.pdf). This short `readme`, derived from that manual, follows the [JOSS][joss] outline for a quick overview of the software. # Examples There is a dedicated complementary package, called `Dataviz`, with examples, that was build with educative purposes, for a recurrent local workshop called the Data Week, where we invite a diverse community in gender, professions, educational and life backgrounds and is automatically installed when you install Grafoscopio. Also we have a Spanish introductory tutorial, that is directed towards beginners. To see such examples please execute this code form the *playground*: ~~~ "This opens the Spanish tutorial" GrafoscopioNotebook open: GrafoscopioDocs tutorial ~~~ After installing the [Dataviz package][dataviz] you can run the following script to see several domain specific applications: ~~~ "This opens the introductory notebook to the Dataviz package" GrafoscopioNotebook open: DatavizDocs introNotebook ~~~ # API documentation Because Grafoscopio inhabits a full live coding environment, it follows the custom of making the API documentation available inside a dynamic environment, instead in some static web page. To open the Grafoscopio package inside the system browser, and see the messages organized by protocols execute: ~~~ "Browser the notebook API" GrafoscopioNotebook browse. "Browse the document tree API" GrafoscopioNode browse. ~~~ # Tests The core functionality of Grafoscopio document tree model is tested. If you want to see and run the tests, just open the tests package by executing: ~~~ GrafoscopioNodeTest browse ~~~ From there, you can use the extensive test environment provided by Pharo. For a short introduction to how to run test on this environment, I recommend to watch the [Coding a Little Domain Specific Language video ][mooc-w3-14], from the [Pharo MOOC][mooc]. # Known bugs There are not known critical bugs now. But the non saved documents don't show any advice if you close them. This is a big usability issue and as we migrate to the new graphical interface, called Spec2, we hope to improve greatly on usability. !!! WARNING Double check you have saved your notebooks before closing them. We are going to hunt and squeeze that bug out of existance. Resistance is futile. To help us with this or other bugs please look at the Community Guidelines to know how to contribute to the project. # Community Guidelines ## Seek support Grafoscopio has a small and new born community. You can reach it by following the contact links in the Grafoscopio page in [Spanish](http://mutabit.com/grafoscopio/) or in [English](http://mutabit.com/grafoscopio/index.en.html). Also you can discuss issues related with Grafoscopio in the [Pharo users community](http://pharo.org/community) mailing list. We are in such list and try to be active participants there and bridge the local Spanish community with the international one. ## Report issues or problems [top][top] To report issues or problems with the software and/or its documentation please visit our [ticket section][grafoscopio-tickets] Fossil repository. Before creating a new ticket, please be sure to visit the [current tickets][grafoscopio-tickets-current], to see if your issue/problem has been not reported already. ## Contribute to the project As we said, Grafoscopio wants to help in blurring the distinction between software developer and interactive document author, so we are pretty open to several ways of contribution: from bug reports, as explained above, to the creation of interactive documentation, domain specific languages (DSLs) and visualizations, or software functionality. Contributions usually take part on our recurrent [Data Week][dataweek] hackathon/workshop and there you will learn how to use and adapt the software, starting by the basics, creating DSLs and crafting visualizations and integrating them into interactive notebooks. You will also learn how to use Fossil and how to commit to our shared repositories for [code][grafoscopio-sthub] and for [documents and issues][grafoscopio-fossil]. Besides this manual, we are creating also a tutorial (in Spanish) with all these themes covered, as memories for us and others to remember and learn from. The idea, as was said before, is to have multilingual documentation with a *local first* approach. If you don't have the chance to assist to one of our face to face learning workshops and hackathons or to use the resulting notebooks, but still want and already know who to contribute, you can also ask for permisions in the respositories using any of the contact methods listed above. We are a small, new born and friendly community with low traffic mail communication and can discuss about contributions on an individual case by case approach, so your words, bugfix and suggestions will be listened and taking into account and integrated when and where they make sense. Welcome again to our community :-). # Licenses Grafoscopio and its tutorial is licensed under MIT license and the readme and the User Manual, and other documentaiton is licensed under a modified P2P license. To see a full copy of such respective licenses, please visit the files under this repository: - [Grafoscopio MIT License](./Docs/En/Licenses/grafoscopio-mit.md) - [Documentation P2P License](./Docs/En/Licenses/documents-p2p-ismb.md) [^ide]: IDE: Integrated software Development Environment [^book-program]: The idea of the "document is a program" is a paraphrasis of "the book is a program", stated in the Pollen documentation, which is a short phrase to express a powerful idea about burring the disctinction between the document and the program, that is present in several programs, like TeXmacs, Leo, Org Mode, and, of course, Grafoscopio. [agileviz-quickstart]: https://dl.dropboxusercontent.com/u/31543901/AgileVisualization/QuickStart/0101-QuickStart.html [beaker]: http://beakernotebook.com/ [bibtex]: https://en.wikipedia.org/wiki/BibTeX [chocolatey]: https://chocolatey.org/ [citezen]: http://people.untyped.org/damien.pollet/software/citezen/ [d3js]: https://d3js.org/ [dataweek]: http://mutabit.com/dataweek/ [dataviz]: https://mutabit.com/repos.fossil/dataviz/ [etherpad]: https://en.wikipedia.org/wiki/Etherpad [flare]: http://flare.prefuse.org/ [floor-function]: https://en.wikipedia.org/wiki/Floor_and_ceiling_functions [fossil]: http://fossil-scm.org/ [fuel]: http://rmod.inria.fr/web/software/Fuel [full-manual]: ../../uv/Docs/En/Books/Manual/manual.pdf [joss]: http://joss.theoj.org/ [grafoscopio-en]: http://mutabit.com/grafoscopio/index.en.html [grafoscopio-fossil]: http://mutabit.com/repos.fossil/grafoscopio/ [grafoscopio-tickets]: http://mutabit.com/repos.fossil/grafoscopio/ticket [grafoscopio-tickets-current]: http://mutabit.com/repos.fossil/grafoscopio/rptview?rn=1 [grafoscopio-sthub]: http://smalltalkhub.com/#!/~Offray/Grafoscopio [git]: https://en.wikipedia.org/wiki/Git [github]: https://en.wikipedia.org/wiki/GitHub [gogs]: https://gogs.io/ [gt-tools]: http://gtoolkit.org/ [hackbo]: http://hackbo.co/ [json]: http://en.wikipedia.org/wiki/JavaScript_Object_Notation [jupyter]: http://jupyter.org/ [jupyterlab]: http://jupyterlab.github.io/jupyterlab/ [latex]: https://en.wikipedia.org/wiki/LaTeX [leo]: http://leoeditor.com [light-markup-languages]: https://en.wikipedia.org/wiki/Light_markup_language [manual-tip]: http://mutabit.com/repos.fossil/grafoscopio/dir?tip&name=Docs/En/Books/Manual [markdown]: http://pandoc.org/MANUAL.html#pandocs-markdown [mooc]: https://siemenbaader.github.io/PharoMooc-website/ [mooc-spotter1]: http://rmod-pharo-mooc.lille.inria.fr/MOOC/Videos/W3/C019-Videos-Spotter1-BrowsingAClassWithSpotter-V2-HD_720p_4Mbs.m4v [mooc-spotter2]: http://rmod-pharo-mooc.lille.inria.fr/MOOC/Videos/W3/C019-Videos-Spotter2-Categories-V2-HD_720p_4Mbs.m4v [mooc-w3-14]: http://amara.org/en/videos/04UWGMwnrdXz/info/rmod-pharo-mooclilleinriafrc019-videos-redo-dsl-hd_720p_4mbsm4v/ [neojson]: https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/NeoJSON/NeoJSON.html [nix]: http://nixos.org/nix/ [nosql]: https://en.wikipedia.org/wiki/NoSQL [nteract]: https://nteract.io/ [oop-pharo-2017]: https://ci.inria.fr/pharo-contribution/view/Books/job/LearningObjectOrientedProgramming/98/artifact/book.pdf [org-mode]: http://orgmode.org/ [pastebin]: https://en.wikipedia.org/wiki/Pastebin [pandoc]: http://pandoc.org/ [pharo]: http://pharo.org/ [pharo-download]: http://pharo.org/download [pollen]: http://docs.racket-lang.org/pollen/ [processing]: https://processing.org/ [raphaeljs]: https://dmitrybaranovskiy.github.io/raphael/ [roassal]: http://agilevisualization.com/ [rolling-release]: https://en.wikipedia.org/wiki/Rolling_release [spec]: http://files.pharo.org/books/spec-tutorial/ [spotter]: http://gtoolkit.org/#spotter [sqlite]: http://sqlite.org/ [ston]: https://github.com/svenvc/ston/blob/master/ston-paper.md [texmacs]: http://texmacs.org/ [top]: # [udbc]: http://www.smalltalkhub.com/#!/~TorstenBergmann/UDBC [zeppling]: http://zeppelin-project.org/ [zotero]: https://www.zotero.org/ [zotero-manual]: https://www.zotero.org/groups/diseo_y_creacion_phd_msc_universidad_de_caldas/items/collectionKey/PHGTCZVT [zn]: http://zn.stfx.eu/zn/index.html