Increase your software development productivity 100x

Get more work done, for less effort!

The above might sound like an exaggeration. However, it is not! Simply because in Phosphorus Five, you can reuse almost everything you do. In the video below, I am demonstrating reducing my projects from 388 LOC (Lines Of Code) to 1 LOC, by creating a reusable extension widget, from an existing web app page.

Basically, what this implies, is that you can reuse almost everything you do in one project, to the next. This suggests that if you’re creating more than one project, then every time you start out a new project, you’re almost done with it, before you even start!

You can find the code for this below the video. For the record, I had to slightly modify the code below, to avoid angle brackets, etc, since my blog can’t handle angle brackets in code segments, even if I HTML encode them manually … (Darn it WordPress!!)

Now before you check out the code, realise that we started out with a “page”. Then we turned that page into a reusable extension widget, in roughly 5 seconds. The end result, being a reusable component, we can reuse in other projects, or in other pages in the same solution. Please watch the video for an explanation of how we did this.

Features of our GitHub project browser

The video also demonstrates some other pretty nifty features. For instance, we are able to traverse any project from GitHub, without using any of the GitHub APIs. The way this works, is that we are able to treat HTML semantically, and extract parts of the HTML returned by GitHub, by transforming it into a lambda object, to retrieve its actual “data”. Basically, we’re “scraping” GitHub, and semantically extracting parts of their HTML, to semantically “understand” the structure of the project we’re browsing.

This allows us to treat HTML semantically, as “data”, and create an Ajax TreeView, which you can see either in the video above, or the screenshot below. The Ajax TreeView in our little “app”, is basically created by semantically “scrape” the GitHub project website, retrieving the data for the links to download its folders, and its files, allowing us to recursively traverse any GitHub project, by treating their HTML semantically.

In addition, we will automatically parse any Markdown files, and show them in HTML view, and show any code segments in a “syntax highlighter widget”. To do this, we first use the HTTP REST capabilities of P5, which allows us to download files, using any of the HTTP verbs, such as GET, POST, PUT, etc. Notice, this is done on the server. Then we inject the code into a “pre” HTML widget (code files), which is later highlighted using the highlight.js JavaScript library. The Markdown files, are first converted into markdown using the [markdown2html] Active Event from Phosphorus Five, for then to be injected as the content into a “div” widget.

To understand how we do this, take a look at the code at the bottom of this blog.

Our little code snippet, also shows how you can use the System42’s Ajax TreeView, in addition to creating a “datagrid” manually, by creating a table element, wrapping your “grid”. Below is a screenshot of our little app.

Below is the code we ended up with, after having created our extension widget. Evaluate the following code in your System42/Executor.

/*
 * Creates a GitHub folder browser, that will allow the user 
 * to browse and read all files within each folder of a project.
 * Will create one Ajax TreeView mirroring the folder structure of the project, 
 * semantically parsed from the HTML of the project's landing page, in addition
 * to a "datagrid" allowing the user to see all files within currently selected folder,
 * and view its content.
 */
create-event:sys42.samples.git-hub.file-browser
  return
    container
      oninit
        p5.web.include-css-file:"https://highlightjs.org/static/demo/styles/rainbow.css"
        p5.web.include-javascript-file:"https://highlightjs.org/static/highlight.site.pack.js"
      events

        /*
         * Downloads GitHub index page, and returns the URLs for both 
         * the child folders, and the Markdown files.
         * Expects [_arg] being URL of GitHub HTML page to download, and semantically parse.
         * Returns [.files] and [.folders] being collection of source files, 
         * and folders recursively inwards.
         */
        sys42.samples.git-hub._get-files-folders

          /*
           * Checking cache to see if requested folder has already been traversed, 
           * and exists in cache.
           */
          p5.web.cache.get:sys42.samples.git-hub.{0}
            :x:/../*/_arg?value
          if:x:/@p5.web.cache.get/*/*
            return:x:/@p5.web.cache.get/*/*/*

          /*
           * Buffers.
           */
          .folders-tmp
          .files-tmp
          .folders
          .files

          /*
           * Retrieving all "xxx.md" types of files, in addition to all folders.
           */
          p5.http.get:x:/../*/_arg?value
          p5.html.html2lambda:x:/-/**/content?value
          add:x:/@.files-tmp
            src:x:@"/@p5.html.html2lambda/**/svg/*/\@class/=octicon octicon-file-text/././+/**/a/*/\@href"
          add:x:/@.folders-tmp
            src:x:@"/@p5.html.html2lambda/**/svg/*/\@class/=octicon octicon-file-directory/././+/**/a/*/\@href"

          /*
           * Massaging results from above.
           */
          for-each:x:/@.folders-tmp/*
            split:x:/@_dp/#?value
              =:/
            add:x:/@.folders
              src:@"{0}:""{1}{2}"""
                :x:/@split/0/-?name
                :"https://github.com"
                :x:/@_dp/#?value
          for-each:x:/@.files-tmp/*
            split:x:/@_dp/#?value
              =:/
            add:x:/@.files
              src:@"{0}:""{1}{2}"""
                :x:/@split/0/-?name
                :"https://github.com"
                :x:/@_dp/#?value

          /*
           * Returning folders and files to caller, making sure we cache results first.
           */
          add:x:/../*/p5.web.cache.set/*/*/items
            src:x:/@.files|/@.folders
          p5.web.cache.set:sys42.samples.git-hub.{0}
            :x:/../*/_arg?value
            src
              items
          return:x:/@.files|/@.folders

        /*
         * Creates our TreeView bugger.
         * Expects [.folders] being initially treeview root items to create.
         * Will also create the default "files datagrid" after TreeView has been created.
         * Hence, also expects [.files], which it will pass into creation of "files datagrid".
         */
        sys42.samples.git-hub._create-tree-view

          /*
           * Deleting any previously created datagrids, if existing.
           */
          if
            fetch:x:/0/0?value
              widget-exists:git-hub-treeview
            delete-widget:git-hub-treeview

          /*
           * Adding our treeview items.
           */
          split:x:/../*/_arg?value
            =:int:8
          add:x:/../*/create-widget/*/*/*/items/*/*/items
            src:x:/../*/.folders/*
          set:x:/../*/create-widget/**/root?name
            src:x:/@split/1?name

          /*
           * Creating our actual treeview.
           */
          create-widget:git-hub-treeview
            parent:results-wrapper
            class:col-xs-12 col-md-6 prepend-top git-hub-tree
            widgets
              sys42.widgets.tree

                /*
                 * Lambda callback for when an item is selected.
                 * We just simply retrieves files and folders, and create our 
                 * Markdown "files datagrid" here.
                 */
                .on-select
                  sys42.samples.git-hub._get-files-folders:x:/../*/items/*?name
                  add:x:/../*/sys42.samples.git-hub._create-files-grid
                    src:x:/../*/sys42.samples.git-hub._get-files-folders/*/.files
                  sys42.samples.git-hub._create-files-grid:x:/../*/items/*?name

                /*
                 * Lambda callback for retrieving child items.
                 * We retrieve the files and folders, and returns items to caller (TreeView).
                 */
                .on-get-items
                  sys42.samples.git-hub._get-files-folders:x:/../*/_item-id?value
                  add:x:/../*/return/*/items
                    src:x:/@sys42.samples.git-hub._get-files-folders/*/.folders/*
                  return
                    items
                items
                  root:x:/../*/_arg?value
                    items

          /*
           * Making sure we initially select the root tree item, and populate our
           * datagrid with the root Markdown files.
           */
          add:x:/../*/sys42.widgets.tree.select-items/*
            src:@"""{0}"""
              :x:/../*/_arg?value
          sys42.widgets.tree.select-items:git-hub-treeview
            items
          add:x:/../*/sys42.samples.git-hub._create-files-grid
            src:x:/../*/.files
          sys42.samples.git-hub._create-files-grid

        /*
         * Creates the "datagrid" that shows our files.
         * Expects [.files] being collection of files to create our "datagrid" around.
         */
        sys42.samples.git-hub._create-files-grid

          /*
           * Deleting any previously created datagrids, if existing.
           */
          if
            fetch:x:/0/0?value
              widget-exists:git-hub-datagrid
            delete-widget:git-hub-datagrid

          /*
           * "Databinding" our datagrid.
           */
          apply:x:/../*/create-widget/*/*/*/*/tbody/*/widgets
            src:x:/../*/.files/*
            template
              tr
                onclick

                  // Databound above.
                  {_url}:x:?value
                  sys42.samples.git-hub._create-file-view-widget:x:/@_url?value
                  set-widget-property:x:/../*/_event?value
                    class:active
                  p5.web.viewstate.get:sys42.samples.git-hub.active-row
                  if:x:/@p5.web.viewstate.get/*?value
                    and
                      fetch:x:/0/0?value
                        widget-exists:x:/@p5.web.viewstate.get/*?value
                    delete-widget-property:x:/@p5.web.viewstate.get/*?value
                      class
                  p5.web.viewstate.set:sys42.samples.git-hub.active-row
                    src:x:/../*/_event?value
                widgets
                  td
                    {innerValue}:x:?name

          /*
           * Creating our actual datagrid.
           */
          create-widget:git-hub-datagrid
            after:git-hub-treeview
            class:col-xs-12 col-md-6 prepend-top git-hub-grid
            widgets
              table
                class:table table-hover
                role:button
                widgets
                  thead
                    widgets
                      tr
                        widgets
                          td
                            innerValue:"Files"
                  tbody
                    widgets

        /*
         * Creates the "file-view" widget that shows the content of our files.
         * Expects URL to file to display as [_arg].
         */
        sys42.samples.git-hub._create-file-view-widget

          /*
           * Deleting any previously created markdown widgets, if existing.
           */
          if
            fetch:x:/0/0?value
              widget-exists:git-hub-markdown
            delete-widget:git-hub-markdown

          /*
           * Massaging URL.
           */
          replace:x:/../*/_arg?value
            src:"https://github.com"
            dest:"https://raw.githubusercontent.com"
          replace:x:/@replace?value
            src:"/blob/master/"
            dest:"/master/"

          /*
           * Retrieving Markdown file, creating HTML out of it, 
           * and creating our result widget showing file.
           */
          p5.http.get:x:/@replace?value
          if:x:/@p5.http.get/*/result/*/Content-Type?value
            !~:text/plain

            // We only handle text types of files.
            sys42.windows.info-tip:File is not a text file.
            return

          // Checking type of file.
          if:x:/@replace?value
            ~:.md

            // Markdown file.
            index-of:x:/@replace?value
              src:/
            split:x:/@replace?value
              =:x:/@index-of/0/-?value
            markdown2html:x:/@p5.http.get/**/content?value
              root-url:{0}/
                :x:/@split/0?name
            create-widget:git-hub-markdown
              parent:results-wrapper
              class:col-xs-12 prepend-top git-hub-markdown
              element:div
              innerValue:x:/@markdown2html?value

          else

            // Source code file.
            p5.html.html-encode:x:/@p5.http.get/**/content?value
            create-widget:git-hub-markdown
              parent:results-wrapper
              class:col-xs-12 prepend-top git-hub-markdown
              element:pre
              oninit
                p5.web.send-javascript:@"hljs.highlightBlock(p5.$('{0}').el);"
                  :x:/../*/_event?value
              innerValue:x:/@p5.html.html-encode?value

      widgets

        /*
         * This is our input for having user supply the URL to some GitHub project website.
         */
        container
          class:input-group
          widgets
            span
              class:input-group-addon
              innerValue:URL
            void:url
              placeholder:GitHub project's URL ...
              class:form-control
              accesskey:U
              value:"https://github.com/"
            container
              class:input-group-btn
              widgets

                /*
                 * Button will download main GitHub project's landing page, 
                 * and semantically parse the HTML for any sub-folders and all of its source files.
                 * Will use this result for creating a "folder Ajax TreeView" to allow you to browse
                 * the project structure, in addition to a "files datagrid" allowing you 
                 * to view the content of each file.
                 */
                button
                  class:btn btn-primary
                  innerValue:Fetch
                  onclick

                    /*
                     * Clearing out any previous results.
                     */
                    clear-widget:results-wrapper

                    /*
                     * Retrieving root URL for project, and fiding all "xxx.md" types of files, 
                     * in addition to all folders.
                     */
                    get-widget-property:url
                      value
                    sys42.samples.git-hub._get-files-folders:x:/@get-widget-property/*/*?value

                    /*
                     * Verifying that we actually found something worth looking at.
                     */
                    if:x:/@sys42.samples.git-hub._get-files-folders/*/*
                      not
                      sys42.windows.info-tip:No folders or files found at specified URL
                        class:info-window info-window-error
                      return

                    /*
                     * Creates our Ajax TreeView for navigating folders.
                     */
                    add:x:/../*/sys42.samples.git-hub._create-tree-view
                      src:x:/@sys42.samples.git-hub._get-files-folders/*(/.folders|/.files)
                    sys42.samples.git-hub._create-tree-view:x:/@get-widget-property/*/*?value

        /*
         * Used to hold our results widgets.
         * Both the TreeView, DataGrid and markdown/code widgets are appended as children into this bugger.
         */
        container:results-wrapper

Then create a “lambda” page in the CMS of System42, with the following code in it, to consume the widget we created above.

/*
 * This will create a page, consuming our newly created 
 * extension widget.
 */
create-widget
  parent:content
  class:col-xs-12
  widgets
    sys42.samples.git-hub.file-browser

Notice, this code example, like most other examples here on my blog, requires both Phosphorus Five, in addition to System42.

PS!
If you want to use the above as a “persisted extension widget”, you’ll need to stuff it in a Hyperlambda file, and make sure your file is evaluated as the server is started.

PPS!
This code is dependent upon the very latest release of both Phosphorus Five and System42 to function, version “Zen 2.1” that is.

KUDOS

KUDOS to highlight.js; I want to give som KUDOS to the guy who created highlight.js, which is a great JavaScript library, allowing anyone to easily show code as syntax highlighted in their web projects. Thank you Ivan Dementev for Open Source licensing your great library. Great work dude! 🙂

In addition, I want to give KUDOS to the Knagis for sharing his brilliant CommonMark.NET library with the world! Great work! For the record, I have a feature request for you, which is to have the option of having a separate URL resolver for images and hyperlinks. Pliiis …? 🙂

2 thoughts on “Increase your software development productivity 100x

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s