The structure of the code in TidyBlocks is dictated by three things:
Rather than describing these at a high level,
let's have a look at a simple pipeline that loads a data set,
and generates a plot:
./index.html is the home page for the TidyBlocks site.
It defines a class called ReactInterface that extends the generic UserInterface class
with all the things we need to make Blockly, React, and our own code play together nicely in the browser.
This class is meant to be a singleton,
only one instance of it is ever created.
UserInterface is defined in ./libs/gui.js alongside most of our application's code.
It initializes our blocks and loads some built-in datasets when it is created,
and provides methods to turn the Blockly workspace into code objects and run them.
The blocks themselves live in ./blocks/*.js.
Each file defines a function called setup that should be called exactly once as the application starts up.
The user can now create blocks and click them together to make pipelines.
When she clicks on the "Run" button,
UserInterface.getJSON calls Blockly.TidyBlocks.workspaceToCode,
which calls the functions set up in ./blocks/*.js (recursively) to generate the JSON representation of the workspace.
UserInterface.getProgram parses the resulting string,
then creates an instance of the Restore class (described below)
and uses it to turn the JSON into a tree of runnable objects (also described below).
Finally, UserInterface.runProgram takes those runnable objects
and a freshly-created instance of Env (the runtime environment class)
and asks the program to run itself in that environment.
When it's done,
the interface can get datasets, plots, and log messages out of the Env object and display them.
The Restore class is defined in ./libs/persist.js.
It has one method for each of the five kinds of runnable object in TidyBlocks:
programs, pipelines, transformations, operations, and values.
These objects are represented in JSON as nested lists;
the first element of each list specifies what family the object belongs to
while the second specifies the exact species (e.g., "filter")
and anything else in the list is either settings or nested objects.
Each method in Restore either creates an object of the right kind (for Program and Pipeline)
or looks up a constructor and calls it (Transform, Op, and Value).
Program (in ./libs/program.js) and Pipeline (in ./libs/pipeline.js)
each define a method called run that takes an instance of Env as a parameter and runs some code.
Program handles the dependencies between pipelines
so that a pipeline starting with a join block doesn't run until the things it's supposed to join are available.
Pipeline handles the flow of data between transformations and returns the final result.
Pipelines are comprised of transforms defined in ./libs/transform.js.
Their common parent is TransformBase,
and each one must define a run method that takes an instance of Env as a runtime environment
and an instance of DataFrame (./libs/dataframe.js) to work on.
Transforms may contain operations (./libs/op.js),
which in turn may contain values (./libs/value.js).
The run methods for these objects take individual rows as inputs and produce new values as outputs;
it's up to the enclosing transform to decide what to do with that value.
There are two exceptions to this tidy story:
the report transform and all of the statistics and plotting blocks store things in the runtime environment
so that the user interface can display them after the program has finished running.
Every one of these results must have a unique name.
The React components for the user interface live in ./libs/ui/*.jsx,
and our unit tests live in ./tests/*.js.
npm run build will bundle the React components with ./libs/*.js, ./blocks/*.js, and ./index.js
while npm run test will run the tests
and npm run coverage will produce a report showing which parts of the code were and weren't exercised
(see ./coverage/index.html for the results).
npm run build
npm run test
npm run coverage
All of this is fair bit of code---at the time of writing, the breakdown is:
However, a lot of it is repetitive:
block definitions and arithmetic operations,
are all close siblings even after common code is factored out.
So suppose you decide to add a new block to an existing category.
The steps you have to go through are:
This isn't trivial, but it's less work than Version 1 required,
and we think it's a solid base for future work.
— Greg Wilson / 2020-07-28