Playing around with MIME/PGP lambda web services

In the upcoming release of Phosphorus Five, one of the things I have given great attention to, is its MIME and PGP parts. One of its really nifty use cases is the ability to invoke cryptographically signed and encrypted web services passing in MIME envelopes as the body of your request. This has a lot of interesting use cases, such as making it possible to demand that the invoker of your web service has cryptographically signed his invocation using a “whitelisted” PGP key – Resulting in a web service that can only be invoked by an authorised client. Or to transmit multiple Hyperlambda snippets, that are evaluated in order of appearance. Or to transmit file attachments together with your web service invocation, having access to these file attachments from your own (client) Hyperlambda, etc, etc, etc. If you have no idea what a “lambda web service is”, you can read one of my DZone articles about it here.

In the upcoming 8.4 release I will have a Hypereval “page” snippet, that actually acts as a web service endpoint, encapsulating a lot of interesting features down this alley. Notice, the code examples here requires that you have at least version 8.4 of Phosphorus Five, which is not yet released as of this writing. Anyway, let us look at what we can do, as 8.4 is released.

A simple HTTP POST web service invocation

So let us first create a simple (non PGP) web service invocation. Notice, if you’ve downloaded version 8.4, you’ll need to use e.g. “Peeples” to grant module access to the “/hypereval/lambda-ws” page to the guest account to be able to use this code.

/*
 * Creating a normal lambda web service invocation.
 */
p5.http.post:"http://127.0.0.1:8080/hypereval/lambda-ws"
  Content-Type:application/x-hyperlambda
  content
    +:int:7
      *:int:20
        _:10
    return:x:/@+?value

The above Hyperlambda snippet will create an (unencrypted) HTTP POST request going towards your Hypereval “lambda-ws” page, and pass in a Hyperlambda snippet, that simply multiples 20 with 10, for then to add 7 to the result, and echos the results (207) back to the client. A slightly more complex version can be imagined using the following code.

p5.http.post:"http://127.0.0.1:8080/hypereval/lambda-ws"
  Content-Type:application/x-hyperlambda
  content
    split:Thomas Hansen is pretty cool
      =:" "
    add:x:/+
      src:x:/@split/*?name
    return

The above will result in something resembling the following.

p5.http.post
  result:"http://127.0.0.1:8080/hypereval/lambda-ws"
    status:OK
    Status-Description:OK
    Content-Type:application/x-hyperlambda
    Last-Modified:date:"2018-06-19T13:17:04.998"
    Server:Mono.WebServer.XSP/4.4.0.0 MacOSX
    Date:"Tue, 19 Jun 2018 10:17:04 GMT"
    Cache-Control:private
    Set-Cookie:ASP.NET_SessionId=34F7802887AEA977F431B3DA; path=/; HttpOnly
    Connection:close
    content
      Thomas
      Hansen
      is
      pretty
      cool

Basically passing in Hyperlambda code, that splits a sentence (on the server may I add), as a web service invocation, and returns each word back to caller, as the body of the HTTP response. The reasons why this is pretty safe and secure, is because as I evaluate the Hyperlambda supplied by the client, I am using [eval-whitelist], which allows me to only allow for a (secure) subset of my server’s Active Event vocabulary. Hence, no malicious code constructs are possible to create that somehow changes the state of my server. What vocabulary you want to whitelist is easily configurable.

Encrypting and signing your web service invocation using PGP

This is where it gets interesting. First let’s have a look at some code that does something similar to our above code, with two crucial differences. First of all it encrypts our code. Secondly it cryptographically signs your code using a private PGP key.

/*
 * Creating a cryptographically secured lambda web service invocation.
 */
p5.http.post:"http://127.0.0.1:8080/hypereval/lambda-ws"
  .onrequest
    .p5.mime.serialize-to-stream
      multipart:mixed
        encrypt
          fingerprint:83bfcbe0235f90e0a67bb865863a22266f9ef39a
        sign
        application:x-hyperlambda
          content
            split:Thomas Hansen is pretty cool
              =:" "
            add:x:/+
              src:x:/@split/*?name
            return
  .onresponse
    .p5.mime.load-from-stream

The above code will create a MIME envelope of our “string split” Hyperlambda. Then it will encrypt the code for then to cryptographically sign the message. The code will be cryptographically signed by your server’s PGP key, and it will be encrypted for the PGP key having the fingerprint of “83bfcbe0235f90e0a67bb865863a22266f9ef39a”. The latter of course must match the PGP key of the server you are transmitting your request to. Which you can read about how to acquire here.

This creates a couple of really interesting side effects. First of all, all of a sudden we have a guarantee of that the request originated by the client that had access to the private PGP key that was used to sign the envelope. Since the web service endpoint allows us to create a “white list” of accepted PGP keys, this allows us to only accept invocations from clients we know for a fact we can trust – Assuming your private PGP key hasn’t been compromised somehow of course.

In addition, it becomes literally impossible for an adversary to spy on your web service invocations. For code, this might not be that crucial. However, the MIME features of Phosphorus Five also allows you to transfer data and files, making this “tiny little detail” important. In addition, it also completely eliminates the problem with a “Man In The Middle” acting as a malicious proxy, tricking you into believing you have a secure SSL context, by giving you a fake certificate. In fact, even in the middle of a “Man In the Middle” attack, you can still have a cryptographic guarantee of that you have a securely encrypted communication channel – Assuming your PGP keys are correct though.

It also prevents “tampering” with the message, since if such a thing occurred, the PGP signature would no longer verify, and the invocation would never be evaluated. And since this all happens “automatically” for you, and the code is returned “semantically”, there is zero overhead for you as an application developer to implement this. In fact, here is what is returned from the web service after invocation.

p5.http.post
  result:"http://127.0.0.1:8080/hypereval/lambda-ws"
    status:OK
    Status-Description:OK
    Content-Type:text/html
    Last-Modified:date:"2018-06-19T13:24:19.390"
    Server:Mono.WebServer.XSP/4.4.0.0 MacOSX
    Date:"Tue, 19 Jun 2018 10:24:19 GMT"
    Cache-Control:private
    Set-Cookie:ASP.NET_SessionId=33FDE6A445F8D46F03602E74; path=/; HttpOnly
    Connection:close
    content
      multipart:signed
        signature
          foo@bar.com:bool:true
            fingerprint:83bfcbe0235f90e0a67bb865863a22266f9ef39a
        multipart:mixed
          application:x-hyperlambda
            content
              Thomas
              Hansen
              is
              pretty
              cool
        application:pgp-signature
          Content-Disposition:attachment; filename=signature.pgp
          Content-Transfer-Encoding:7bit
          content:blob: ... a long piece of PGP signature blob ...

At which point you can retrieve your Hyperlambda results with a simple x-pression. If the web service endpoint receives something that is cryptographically signed, it will encrypt the return value for that same PGP key, and cryptographically sign it with its server key. Resulting in that you have a 100% water tight cryptographically secured guarantee of that nobody have been tampering with your request somehow, and nobody has seen neither what was sent, nor what was returned.

Yet again, I want to emphasise that the code to accomplish this is not yet released, but will be released soon in the upcoming 8.4 version. If you’d like to see if 8.4 is released, you can check out this link. For the code examples, I used “Hypereval” myself, doing a simple loopback invocation to my own localhost server. You can of course use anything else as your client though, including CURL or C# code – As long as you’re able to somehow create the correct HTTP requests. And you can of course use anything as a client, and any Phosphorus Five server as a server.

As an additional bonus I have also created a GUI wrapper, allowing you to play around with Hyperlambda constructs, to test your web service code. Below is a screenshot.

When executed, a modal window will be displayed, with the same result and structure a web service invocation would yield.

Advertisements

PGP based server to server communication

Using PGP when two server communicates with each other has a lot of advantages, such as among other things reducing the probability of a “Man In The Middle” attack, by cryptographically signing and encrypting data sent from one server to another.

In the upcoming 8.4 release of Phosphorus Five I have made this much simpler. First of all, when you install your server, you can check of simple checkbox, and have your server’s public PGP key transferred to your configured key server. The default key server used in Phosphorus Five is “keyserver.ubuntu.com”, but this can easily be changed in your web.config file.

Secondly, when some MIME envelope is parsed, and it has been cryptographically signed, Phosphorus Five will automatically retrieve the public PGP from your configured key server, and install it into its PGP context.

Thirdly, I have created lots of meta PGP key retrieval URLs for a default Phosphorus Five installation, allowing a server to automatically communicate and send public PGP keys back and forth. For instance, if you need to securely communicate with a server using PGP cryptography, you can simply request the server’s base URL and append “/micro/pgp” to it. At which point the server’s public PGP key will be returned as ASCII armoured text. Notice, you’ll have to use “Peeples” to explicitly allow for accessing this URL if you wish for non-root accounts to be able to retrieve keys. Requesting my personal development server’s main PGP key for instance returns the following.

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: BCPG C# v1.8.1.0

mQINBFsmpEUBEACPPTzy3Tg95PoQcpY6CetYbC6+1kKgn1ue9M5yMwyx/MA5JIS5
FfiQBXCzyIDTPN1xquvkAM59w0FkBfgI3ZjYojKLkx3DcEjB3acigzjxHRCvdZML
7Q6A0lWgnlV2Uw9KZmP40LHcWRSvq97eGse+wX9G278J2bLWggxVoBFXZqXnduxf
FVLMcXdA1phhk/ClMjsrPnX5tbUCXsE+wj4ZzKA81hkdg1Bq8pLeUWklzGazzUe4
lE2EJ0uekdoiMYT+7ov4H+3hf3T30naWfkR9rSb7e3owBjceJIlao3JvSfQ7jbkc
ZaOSZq1USe87PPLqvhrzNDCZqvs3EUjvXwYtGNfpDr+W/lezqTpDVenmm0g1179u
lZHpXlYkY0wqn8hY0fvjrb065Ah/FHmchzY6e4a8Lh1Iiu92pEBp6TAmdJ+AyaDF
8jsoSi+MZXKjorCmeGfiHNVCUfWL7ZMLEJOgxqpJIWxdyZ6GJOUZLs0skgCM4k5C
vXl9+ojHo+0a9pnj0xKwIsTRtipSvF+UxMgEBOHCeRC+Z9RNm9S06wzDkx2FNdi8
YC5nawSr+H1F21Ijj7nyV+B7Neb088FSnlfLn1dEcREgWhPTfu4wPy0TMzForLrQ
ULakl93CU4A57VH+8gjYuPRBk39Bz5QQhkTXVdub0Hpk4ugfXmx6LmuW6QARAQAB
tDFEdW1teSBUZXN0aW5nIEtleSBOb3QgaW4gQWN0dWFsIFVzZSA8Zm9vQGJhci5j
b20+iQIxBBABAgAbBQJbJqSCAhsDBAsJCAcGFQgCCQoLBQkFpOvFAAoJEIY6IiZv
nvOaYHIQAIvMpAdg5YsZH1cMzFFBQxMeM0hoBJottuszsuujiTyachnV8a2cZX7G
xdek2WtA/ipcuUIJJKppztjB3aNvqqUtYBhXkng0KsxWC8nctDiHNKfxsNnZ/s+4
T1Nq3cjlAH8ihWWPFCjJalwQ5nKyT+zzsVM8nfuIyjoPlZdcjbv+pJSwrsX3388L
C7J1NIxv1HYbGMT23gyPPF/S6k3mqbEbMAmphnKfMWfMDVRep8W8q4dDvPYtG3Tg
v4dwjJxhN3lzlKaSVLAauHPnVyeC44P1NMZQzXdx0Lnhd1r5ibucemxCmMrGGDvZ
/NiktmOCX9H/dykESurjBv820Xpa6r+/Z1gZ/WsaxOw3uqPyH6YgPDkdcDhisGhO
0e6yz2Sfx52VPIAw2K6EkeKLuh+MmbgEI7PZMz/j66VdsJXtqMetXCMbabWgF4V0
Sq64lcbqW0ixIFULQKEVkOCG7Jc+CaGkh1JoVcpE7YZty7wIaBPNaYYA07BhDmSc
Fs/Kb/i5AGW/cEr284k+Dx4oxKEDTmTLSgDdlNiu2Kk2KuyrwbLJPGD/iNWp7c07
Fb1e4WHyCHKaFnaWEaG6qeiLaXSyy4rHOQof0J+7/ArFc92UDtMcUWtHatIcxU0Y
vSYYzNWkLYqOghrkw1W58bGmFKa6Bzx3CzpcoQ2m350Jzhf9O2qG
=QUYz
-----END PGP PUBLIC KEY BLOCK-----

In addition, you can also list all public PGP keys a single Phosphorus Five server has by requesting the url “/micro/pgp/list”, which for my server yields the following (Hyperlambda).


83bfcbe0235f90e0a67bb865863a22266f9ef39a
  id:6F9EF39A
  algorithm:RsaGeneral
  strength:int:4096
  creation-time:date:"2018-06-17T18:11:17"
  is-encryption-key:bool:true
  is-master-key:bool:true
  is-revoked:bool:false
  version:int:4
  expires:date:"2021-06-17T18:10:18"
  user-ids
    :Dummy Testing Key Not in Actual Use <foo@bar.com>
  signed-by
    6F9EF39A:date:"2018-06-17T18:12:18"
d9d9a341717d93ce911958aeddbb618d4f2ac9a9
  id:4F2AC9A9
  algorithm:RsaGeneral
  strength:int:4096
  creation-time:date:"2018-06-17T18:07:40"
  is-encryption-key:bool:true
  is-master-key:bool:true
  is-revoked:bool:false
  version:int:4
  expires:date:"2021-06-17T18:07:03"
  user-ids
    :kgkgiygiugiugiyg iugigiug <igigiyg.ouhouh.no>
  signed-by
    4F2AC9A9:date:"2018-06-17T18:08:03"
... etc ...

… or you can query for specific keys, using a URL such as for instance “/micro/pgp/d9d9a341717d93ce911958aeddbb618d4f2ac9a9”. Which yields the following for my server.

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: BCPG C# v1.8.1.0

mQINBFsmo2wBEACbEOC6qUenvvQqj2uj2RQ+eFfrcOkaQk+KKl/xoY6HDbvbQtw6
5EVx9tsftr1zzznqGT8/M2pHlJQdgaOMg8m7hzdozWkazK1L1BPEoV/r34EuLmDg
WhihMDb28Y3MBV/5Zkl9QRR5FClR2cpy/dTKc0ANwcBpqXcHzdYYviA+SoST96un
kjV85YS77WBdIEHhfBkjjDUBnjG/934y5v1txtvtBrr9MqxDJy4kgR7yHZmh4c1H
8uRkuCxWf9eJ0BH9VXhUDTDD7NTBFAdNSqH7lWfF8P8N4Z+URI6igeLET8JUZBio
5V2LvleESp9C78Sacv5j7LocqW0bL3nki0sma/yIs0OocHZxELi/or3ysW9K3KBd
z6dh/aT4TCkIh7/DuCowtUKmasfFmol/te8q2irIsBqWH32Z+7GZ1UaO9tiok2JF
BGK7+3wQcJFXnKSxppQ01KY46Nw3Pg17CgRuvMCFT+PhmGFKm8mwo2cvkq64NFU0
gjHeY4BG2iSvDLQDNfRWjA5mleUmeRBM8lYw+9Cqq3vOrWSDoLzPt9YKeKsrn3HU
SDs/IsSylIwppkpY+cKZE3zl17Eco3NwvWfWLFrr3eJxY3UsknspwQz10siEL4Z4
+Diz2oqtGPppTUBUZZOPUr1cspy61+czxEnSF+1hMVLY9tnCS12vWAQh5QARAQAB
tC1rZ2tnaXlnaXVnaXVnaXlnIGl1Z2lnaXVnIDxpZ2lnaXlnLm91aG91aC5ubz6J
AjEEEAECABsFAlsmo4MCGwMECwkIBwYVCAIJCgsFCQWk69sACgkQ3bthjU8qyalv
bxAAhwdqydcskiogAlAD1J3LarFIiWATfV5fD469xaoNq8nKLgXTDZesuCfpTY78
a5etBZDEq+UJmaxy/RfujTiDFAB3OT4mFqbToRTkTyCLXVrCivaB8gMTJS7TUNS7
/WmiV2yjSXiTPnQZbR1INgEd8JbiJ8GJbHJqHU6LPIWcSg0sH3CMuDfL6wihugFp
lpAYduDaAEhEPcK2iADcByKy6o+WUVOyivDculXkJY2migPEBhMEHoINjc9ooIep
AdyEq4HLcXPjKPCXWZp4zqX9oTCxX5q0+YC3TS6n4JKZGNMYYgMoDOq2EJ8HA5WR
RtfOq7qDQ/Vw3V2/5DBFYxwmet/U5LMBzQwCdcIXVLbBjmnclSxmbM1dg6mjAsOL
TZ3UP1s//UZB6MOzcufRaWVwFcV4sKZ+h11pwjdXER8isFfMofnze5GqtE1Lm//V
kcWOs4+gvFH3Tb4gjOXi+pIcPKxBPp8YGjSQTI2tO7N789irKATYCrfguHXHEv1V
ZjFwgIPv0NmxqHdVe0WAnjmQaaTqtLMNnUCeIV5ytM/uSKbQwkVesDZUdXgPP3Lp
ryZYKYWMeCE9epjURs3ileMfNEPOtGc27rBQHUaNHD5HG0TxqXgVqWdg1xpVJy+N
jBibMutj/fV3PzEhSUMlVN61/smJDopYBgFOCv4c06JNZbw=
=/BDf
-----END PGP PUBLIC KEY BLOCK-----

You can also of course return multiple keys at the same time, by instead passing in things that will be matched as the “identity” of the key, such as for instance “/micro/pgp/Hansen”, which will return all keys having “Hansen” somewhere within their identity.

All in all, this creates some pretty cool opportunities for secure communication, allowing for meta retrieval, having automated processes retrieve server keys, and such immediately establishing a secure and encrypted communication channel.

I will also implement more of these types of “convenience” methods and functionality before the upcoming 8.4 release, allowing you to do lots of other interesting things too. However, that was that for today 🙂

MIME Magic

In the upcoming release for Phosphorus Five, I have given a lot of attention to everything related to security, which also for obvious reasons includes how P5 creates MIME messages, and are using PGP cryptography to encrypt MIME messages. The MIME features of Phosphorus Five can be found in the project called “p5.mime”, and internally it is using MimeKit. This implies among other things that parsing and creating MIME messages in Phosphorus Five is fast as greased lightning. I have previously tested its capacity myself, and I were able to create a multipart MIME envelope containing 5,000 (small) images, encrypt the whole thing, and save it to disc – And the operation was over after 10-12 seconds on my MacBook Air from 2016, and it spent almost no memory. Decrypting the MIME envelope, and re-creating its files, also took roughly 10-12 seconds.

The sheer speed of MimeKit, and its intelligent usage of streams, hence allows us to use MIME “outside the box”. One of the things I’ll implement wrappers for probably quite soon, is the ability to invoke and handle PGP encrypted MIME web services. This has a lot of interesting advantages, such as eliminating “Man In The Middle” attacks, by expecting the web service invocation to be cryptographically signed, by a trusted client’s private PGP key. In addition to that since it’s integral to the PGP standard to actually compress the resulting encrypted content, also reducing bandwidth consumption. In addition to (of course) making a potential “Main In The Middle” completely in the dark as to what is actually transferred between the web service endpoint, and its clients. And by combining this features with lambda web services, this opens up for a whole range of interesting use cases and new approaches to solving your problems.

Notice, I want to emphasise that some of the features I describe below, are still not yet released, but will be available in Phosphorus Five as of from version 8.4 and up. If you wish to see if version 8.4 has been released, you can check out my GitHub project here. If you want to play with these features after 8.4 is released, you can use “Hypereval” and paste in the code examples I have included below …

MIME is one of those internet acronyms, and it means “Multi-purpose Internet Message Exchange”. Even though most people simply knows it indirectly through their email software, it is highly useful for a *lot* of other scenarios too, and literally a “multi purpose” format. Which I intend to illustrate with some examples at the bottom of this article. However, first let’s try to learn its basics, before we go “all Ninja” with it …

Creating a MIME envelope

To create a simple MIME envelope with Phosphorus Five is dead simple. Use for instance the [p5.mime.create] Active Event, pass in your MIME entities, and VOILA! Below is a minimalistic example illustrating this. Yet again, use “Hypereval” for the code examples in this article.

p5.mime.create
  text:plain
    content:This is your MIME content!

The above results in the following.

p5.mime.create
  result:@"Content-Type: text/plain

This is your MIME content!"

If you want to create multiple MIME messages, you can pass in multiple MIME entities, like the following illustrates.

p5.mime.create
  text:plain
    content:This is your MIME content!
  text:plain
    content:This is your second MIME content!

You can also provide expressions, leading to one or more nodes declaring your MIME envelopes.

.mime
  text:plain
    content:This is your MIME content!
  text:plain
    content:This is your second MIME content!
p5.mime.create:x:/@.mime/*

The above would result in the following.

.mime
  text:plain
    content:This is your MIME content!
  text:plain
    content:This is your second MIME content!
p5.mime.create
  result:@"Content-Type: text/plain

This is your MIME content!"
  result:@"Content-Type: text/plain

This is your second MIME content!"

Persisting MIME envelopes to disc

If you’d rather persist your MIME envelopes to disc, you can use [p5.mime.save]. This event requires a filename as its main argument, and expects the user to have write access to the filename you supply. Below is an example of using it.

p5.mime.save:/foo.mime
  text:plain
    content:This is your MIME content!

The above code will create a file named “foo.mime” in your server’s root folder, containing your MIME envelope. Notice, you can supply multiple MIME entities to your [p5.mime.save] event. However, this will simply persist your MIME envelopes consecutively into the same file, which is highly likely *not* what you want. Hence; “Multiparts to the rescue.”

The whole idea with MIME is that it’s actually a tree structure, or a graph object. And by creating a “multipart” MIME envelope, you can wrap multiple MIME entities into the same MIME envelope. Below is an example of creating a file on disc with a multipart containing two text MIME entities.

p5.mime.save:/foo.mime
  multipart:mixed
    text:plain
      content:This is your MIME content!
    text:plain
      content:This is your OTHER MIME content!

The above will result in a file resembling the following.

Content-Type: multipart/mixed; boundary="=-nqxGFPSK1IvoMUwBVqzf2g=="

--=-nqxGFPSK1IvoMUwBVqzf2g==
Content-Type: text/plain

This is your MIME content!
--=-nqxGFPSK1IvoMUwBVqzf2g==
Content-Type: text/plain

This is your OTHER MIME content!
--=-nqxGFPSK1IvoMUwBVqzf2g==--

The above is actually a tree structure, with one “multipart” containing two “text” parts. This feature of MIME allows us to embed multiple objects into a single MIME “envelope”.

Notice though that since the main argument in [p5.mime.save] is the path to your file, you cannot supply an expression leading to your MIME envelopes the way we did with [p5.mime.create]. You can though supply a path as an expression. Notice also that if you use [p5.mime.save], your MIME envelopes are serialised directly into the stream, never consuming much memory on your server, allowing you to create MIME envelopes in the gigabyte size, without exhausting your server’s resources. You can of course nest as many “multipart” MIME entities as you wish, creating entire tree structures, encrypting and cryptographically signing parts of your MIME entities, with different PGP keys too – If you wish …

Encrypting your MIME envelopes

This is where it gets really interesting. The PGP features of MimeKit, and hence Phosphorus Five, is really quite stunning. If we wanted to encrypt the above multipart for instance, we could easily do so with a couple of additional lines of code.

Background; When you installed Phosphorus Five and started it for the first time, Phosphorus Five would spend some 20-40 seconds during the initial installation process. This was the place where you had to choose a “root” password, and supply your name and email address. The reasons for this, is because it will create a PGP key pair, which is used for encrypting your “auth” file. This PGP key is your server’s PGP key pair, and can (obviously) also be used for other things too. Such as for instance encrypting your MIME envelopes. Below is an example of encrypting our “multipart” from above. To evaluate this code, you’ll have to replace the [fingerprint] value with the fingerprint for your own PGP key – Which is explained further down in this article.

p5.mime.save:/foo.mime
  multipart:mixed
    encrypt
      fingerprint:E43BB8C3A3B9E8DE980051A3256D5F454989255B
    text:plain
      content:This is your MIME content!
    text:plain
      content:This is your OTHER MIME content!

The above would result in a file containing the following.

Content-Type: multipart/encrypted; boundary="=-s8kN2YhaHGekbhDQlp62PQ==";
	protocol="application/pgp-encrypted"

--=-s8kN2YhaHGekbhDQlp62PQ==
Content-Type: application/pgp-encrypted
Content-Disposition: attachment
Content-Transfer-Encoding: 7bit

Version: 1

--=-s8kN2YhaHGekbhDQlp62PQ==
Content-Type: application/octet-stream
Content-Disposition: attachment

-----BEGIN PGP MESSAGE-----
Version: 

hQIMAyVtX0VJiSVbAQ/+NfMak/aAqk88ybPXT4/Z0kmKy0t4rN6M3vl5duPnsoaq
ybAF/J6JOvzJkbgnelnTBuH18mvYCvRlUqb91trUZkGEiQ69i22LJPJlilIE/lWh
E238K2WcDe7bp6Jd3aQnwRUbhxF++Ej2r7SgF46JfaYga7GPQEtTLD5vv0bupU1r
YD96IhFkn8zciDRzzUFJpBfjJqfH3/AD/m4QXFv72u/PSqpCZ4JhxDe2OWn3UjOm
D8HYUsj38zLqDr5Yu186/bKWyGN02Ma7NBJfJuTU8bdYbC+vn7Xi4vOhT94Gm2zs
TS6X77vr4MIMVyxIpbJd4ZOpgR0Od5VIZK643w2k1QPb2ZjTxrCNFlveX6AsFAbG
S/L8muQ9cnyvalgrkZ/WjhoKgXc3zJTFGoo9ssJAjr4+wV8oFB10/03WOs2K05b5
GCzGFO/gz4oNEt2hS4kiE+j4J2lRTukQivie4P66zu6o6iA+LbftypWcHn2WtCvP
T1ChBKz1q+q9OlRngZsPU3dIPrfoUNX4EoAjx1LCel1FnUnHmlIMilrpL1Cr2S4I
ReUjHzgRL/qeQlmOh4MWIk9SeHYDMNifCHOkrH+ULyuht2ySe6LdIifZDQj0X27R
fu4sqlJ5XPJLzepp/2E0f/v9S6ew+/2k2I492yhpOdp1ebCnpDkxjb0ZRy92IiTS
uwHsFWLAca9X0GDNRWOZL7oqhcOiqv20FtXJFma38onfsESuu0yhgL9sSogEFekh
rbneKuZvD4deAOObRmWghd1uq2boVysmQuc7t+20D5k9ZZNXzrcVDJfbBJiHseM+
N2liJAL+jWlZenAFYGwehvIH9OKi9kdfmsyXVxxH4Oj80sI6nrECb4FwGtv+x89F
AyoiMR+Tdj9Ja0BW0LbNen+NTxm+gLDnAnyxGUYU9focis2s0WUFqo5e480=
=EzVG
-----END PGP MESSAGE-----

--=-s8kN2YhaHGekbhDQlp62PQ==--

This is actually how your “auth” file is persisted to disc, and the above will encrypt your MIME multipart, such that only you (having access to your server’s private PGP key) can decrypt it.

Notice, you’ll need to exchange the above [fingerprint] argument to [p5.mime.save] with your server’s actual fingerprint. You can find your server’s PGP fingerprint by for instance opening up your “auth.hl” file in for instance Hyper IDE, and see the first line in it – OR better, you can use the [p5.auth.pgp.get-fingerprint] Active Event to retrieve it dynamically. Which is probably the preferred way to retrieve it, since hardcoding the fingerprint into your code, would not make your code portable. Below is an example of dynamically retrieving your server’s PGP fingerprint to accomplish the same as above.

p5.auth.pgp.get-fingerprint
p5.mime.save:/foo.md
  multipart:mixed
    encrypt
      fingerprint:x:/@p5.auth.pgp.get-fingerprint?value
    text:plain
      content:This is your MIME content!
    text:plain
      content:This is your OTHER MIME content!

And in fact, if you do not supply a fingerprint, but still provide an [encrypt] argument, your server’s fingerprint will be assumed, allowing you to easily encrypt your MIME envelopes with the following code.

p5.mime.create
  multipart:mixed
    encrypt
    text:plain
      content:This is your MIME content!
    text:plain
      content:This is your OTHER MIME content!

Cryptographically signing your MIME entities

If you want to cryptographically sign your MIME entities, this is equally easy as encrypting them. This has a lot of advantages, allowing you to create a guarantee of that the MIME envelope has not been tampered with in any ways, and originates from the party having access to the private PGP key that was used to sign the envelope. This hence gives you a cryptographic guarantee of that nobody has in any ways “tampered” with the message somehow, from the time it was created by the private PGP key that was used to sign it, to the time you received it. Below is an example.

p5.mime.create
  text:plain
    sign
      fingerprint:E43BB8C3A3B9E8DE980051A3256D5F454989255B
        password:ThisIsNotAGoodPassword!!
    content:This is your OTHER MIME content!

The above will produce something resembling the following.

p5.mime.create
  result:@"Content-Type: multipart/signed; boundary=""=-A4YJFmKDqqYVy1NO0smuKg=="";
	protocol=""application/pgp-signature""; micalg=pgp-sha256

--=-A4YJFmKDqqYVy1NO0smuKg==
Content-Type: text/plain

This is your OTHER MIME content!
--=-A4YJFmKDqqYVy1NO0smuKg==
Content-Type: application/pgp-signature; name=signature.pgp
Content-Disposition: attachment; filename=signature.pgp
Content-Transfer-Encoding: 7bit

-----BEGIN PGP MESSAGE-----
Version: 

owJ4nAEfAuD9iQIcBAEBCAAGBQJbJhWKAAoJECVtX0VJiSVbozQP/ikZNPnNhB3w
I5A9d+h5Qn+rVJToWfRczhbpl22VymYLeKsLCrLgmSlZxGa/B8PYRwKD6FkYF6aV
nQ7QkMacnwUN429MfwhRJYSMzwRBF4q/Vdd8qaT2WEvSlEmbHg4NeVNLa/IRAnII
KNcAcS7IenILM0IBRX6CKFzPicZL9AAiNwR2x+u45+Ztzs2Rt3Z/5pnXmosO+47D
5+iTfEZJEIrAwELTSMYA/Si+9hQPTICJ5Yc+gcGSzg5JjjKHEGLodzzSxLb1bhwR
8mdUw1gktUaQscZp3kTSGUI8ZYscEjPlsGjSEwZQ9rX16guvHTEjs2qgxd2eP/4t
MIF9W22C86Bf5ffxPwS+7ldJEnaw70Q1geg1hA6K9lfHb3ckaTDdl8RJPE4KcQSv
JaE5ypr9d1LupDjtAN32V9ZSKNGwSU8gz1amx14eoQthwKRVe1kCS6j3DWGWJCbS
B4N/eLG8unkuOiCPZRz2PHGtug1dHXh4p3W7XptEcSTtDeFP0JTWawMaucS8CWNN
+7sgWm7yFxu055vhynGz7J3gGirjeLl8rJctTuXnlU9XCs/eqi/V9H2AtATeUshx
jiX90WtWbH+IaYBI3rMHWTIS7JBSpLZvuO1Kugau+TRKNFTfa21CWeSKUfgHX9QC
NAgFEWI5ytAmMufkbVXVjUcYhPTx94QTevz/BA==
=Vz1E
-----END PGP MESSAGE-----

--=-A4YJFmKDqqYVy1NO0smuKg==--
"

The PGP signature above will be verified when the MIME envelope is parsed, and if its signature is not verified, the MIME envelope will not be accepted. If as much as a single byte inside of your “text/plain” parts above has been tampered with, the signature will not be valid, and Phosphorus Five will not accept the MIME envelope. And since the signature is created with your private PGP key, and can be verified with your public PGP key, this creates a guarantee of that only the entity with access to the private PGP key was able to construct the above MIME envelope. While allowing anyone with access to your public PGP key to verify its validity.

Notice the [password] parts above. Since Phosphorus Five stores all private PGP keys synchronously encrypted, in the same format that GnuPG stores your keys – This implies that you’ll need to supply a [password] argument, in addition to your [fingerprint], to be able to sign your envelope. Yet again you can use a more dynamic approach to fetch your server’s PGP [fingerprint] and [password] such as illustrated below.

p5.auth.pgp.get-fingerprint
p5.config.get:gpg-server-keypair-password
p5.mime.create
  text:plain
    sign
      fingerprint:x:/@p5.auth.pgp.get-fingerprint?value
        password:x:/@p5.config.get/0?value
    content:This is your OTHER MIME content!

…or completely omit both the [fingerprint] and [password] parts, at which point your server’s PGP fingerprint and its password will be assumed.

p5.mime.create
  text:plain
    sign
    content:This is your OTHER MIME content!

You can of course both encrypt and sign your envelopes, using different fingerprints too if you wish. This allows you to encrypt an envelope for some recipient’s PGP key, while cryptographically signing it with your own private PGP key. This creates a guarantee of that the envelope originated from you, and that it hasn’t been tampered with – While only allowing the rightful recipient to decrypt it and see its content. And if you somehow receive a MIME envelope that has been cryptographically signed, Phosphorus Five will automatically download the public PGP key used to sign the message from a public PGP key server, if it does not have it from before. The last detail allows you to receive a cryptographically signed MIME envelope, for then to immediately start encrypting another MIME envelope to return to the sender. Immediately having established a secure communication channel. Both for authentication and encrypting the content sent back and forth.

Parsing MIME envelopes

So far we have only created MIME envelopes. Let’s look at how to parse them. This is equally easy as creating them, and can be done by for instance using the [p5.mime.parse] Active Event. Below is an example.

p5.mime.create
  text:plain
    sign
    encrypt
    content:This is your OTHER MIME content!
p5.mime.parse:x:/@p5.mime.create/0?value
  decrypt
    password:ThisIsNotAGoodPassword!!

Yet again, you can omit the [password] above, at which point your server’s PGP password will be assumed. And if you’re in the highly “paranoid” spectrum, you are of course free to change your PGP key’s AES password, which must be done before you install Phosphorus Five, by editing your web.config file. The above invocation to [p5.mime.parse] will result in something resembling the following.

p5.mime.parse
  multipart:encrypted
    signature
      thomas@gaiasoul.com:bool:true
        fingerprint:E43BB8C3A3B9E8DE980051A3256D5F454989255B
    text:plain
      content:This is your OTHER MIME content!

Notice how the value for the above [thomas@gaiasoul.com] node is boolean true. This implies that the cryptographic signature has been verified, and that the message has not been tampered with in any ways. If it has a value of “false”, you cannot trust the message! Completely omitting all passwords and fingerprint arguments from above, to rely upon the server’s main PGP key, could be done by constructing your code like the following.

p5.mime.create
  text:plain
    sign
    encrypt
    content:This is your OTHER MIME content!
p5.mime.parse:x:/@p5.mime.create/0?value

The above works due to that if the message is encrypted, and no [password] is supplied, your server’s PGP password will be assumed, and the MIME envelope will be decrypted using your server’s PGP password, when the PGP key needed to decrypt the envelope is released from GnuPG. You can of course encrypt, sign, and decrypt multipart envelopes, nested in as many layers as you wish for that matter. Below is an example of creating and parsing a more complex MIME envelope.

p5.mime.create
  multipart:mixed
    text:plain
      sign
      encrypt
      content:Some MIME text part, this was encrypted and signed!
    text:plain
      sign
      content:Another MIME text part, this was only signed!
    multipart:mixed
      application:x-hyperlambda
        sign
        content
          foo:bar
            Hello:World
            this-was:cryptographically signed!
      text:plain
        content:Jo dude! This was not signed, nor encrypted!
p5.mime.parse:x:/@p5.mime.create/0?value

The above code will create a complex MIME tree structure with a multipart mixed as the root entity, containing two text parts, and another nested multipart, where some of your entities are signed, some are encrypted, and some are both encrypted and signed. You are of course also free to encrypt and sign your different entities with different PGP keys as you see fit. Allowing you to “triple encrypt” an envelope, with three different PGP keys, implementing arguably the “onion protocol” for web services, or emails if you wish …

Notice how we are able to directly supply “application:x-hyperlambda” above, and inject Hyperlambda directly as a node structure. This is a convenience feature, allowing you to easily transfer Hyperlambda in your MIME envelopes, and automatically have it transformed into a lambda structure on the receiving end – Assuming the entity that is parsing your MIME envelope has this feature. After executing the above, your result will resemble the following.

p5.mime.create
  result:@"Content-Type: multipart/mixed; boundary=""=-1hQmuiWvF8ISxBVeLWnprg==""

--=-1hQmuiWvF8ISxBVeLWnprg==
Content-Type: multipart/encrypted; boundary=""=-SS7K3W6vwN1PO8Q4J5EpCg=="";
	protocol=""application/pgp-encrypted""

--=-SS7K3W6vwN1PO8Q4J5EpCg==
Content-Type: application/pgp-encrypted
Content-Disposition: attachment
Content-Transfer-Encoding: 7bit

Version: 1

--=-SS7K3W6vwN1PO8Q4J5EpCg==
Content-Type: application/octet-stream
Content-Disposition: attachment

-----BEGIN PGP MESSAGE-----
Version: 

hQIMAyVtX0VJiSVbARAAiyRdBIPagxCPjh+Ecnj+cogBVFvG0icGJzwX2gjfuEMP
WgOojNr66i0FiCR2fZcrWpxZ79c6Z0tYNuUq+Nr90cEYvx5pPBenfrLG8XfjTJqJ
0lQvkaSDzG0E/bc327Nue6BuYnJhfrLIVj6GZ0dj3ilzX0ly+W481i/wpPDEiM6Q
dlXeLpsyvRDg6gdC/ADbtGj85sA2v0m38vEkApn9Fapg8hKusDYj2c/Ek2OCxpyZ
NCE3xUuD27yvk7DzS7qVUeJEAujulhSlyKMNhsdtUPduBXo9Wv63IdpsfxWUvVOc
8ACO8ieMrYWHS3+paiX690MrFrayySexUwWqXRD/eLY/Cc0KJGNbw2xIhqqxdjxr
Z0Ht3morKl04IcWTosZsNhWa6bxjMfbERWafeIa20d71HHDmz//hSv9vI5xa+Cjy
/3ELrQ+ns7mrdH05qYHaUIvdPKgwvmSeSoSa0fQXn2BHFRNaA5qUsrQvP8HcsFob
vPvpemZ7f0mdrdZ20rFgirVE28DpWNjODzK6RUJYMV2zoVYhXkNK8vp0WEiHSOnz
VNSdk0pQeniOOC57m1rXSG/6TbopZh2yKsHS3v7q5mh1eK1Yxf8ZiOOe756WZsJI
7ySUzYfKDnjOHNPOSZdtyBNuetxCvVtmP0ph+ff0hYHed/edNwk0gq4al1cSofzS
whUB3QhdtUpxVT1CKPSC/KmyrREIVV0k6/99NBeDlw7fPjEzVDCG5JDcOBGmfZ9S
3Y1oWRMdaS7kC+bl1NBLEeMx29/cmzrfMNNrIQ22FVYPglrl6QPjt7vhnlr7ta5y
tSf4whrMI6jDhgP0wszgnYDCZm1ers6KsZ2VWvU/CsCDQUAZP4L3PdkxYPWZ2kji
7t2Lhsy74rt76robh5XR+fI1rPCHdKmOPR8DHs7w9lbeHDugJNRE6ZvXrIEY7SWx
Dki50f3KzJVRO0pdo8PCSAxzM5PGi15ZMsE7Hy6lykt82Fkq3dIHHXE9mwu++e5d
ElIkZvrnQxpvRCrQWwRpnUeVPgENbx+3/aA/q92NZ3u1CGJL3eIe1Sw1QqhHNhx2
TOudluBUb+Rm5S9aRHuvvlvmL3ItSVbl1FXnBID/I5IfORMq1Zq7BinfVn2+/Az1
VssVX1T6JZKR7B9LV7fhENsR0XD+1jtMt9MKtByyyAxDhMZxXBhz4uzmIBZNEJvb
Lu+uxTOAYXpEVYEuYWL4ZHVdeZvJeTTwVw4Vm6mIH+tiqX1JdO/C5jAv82/wmsl1
qHA5Kua0sLI3r9juOhNRKP59vfN6n8p74BNpwTDUsbPhGklfEuy7bhJrVZKPn7Wy
pFLONf2XKMdg8wneLVAmu8oNrcPzVr5x3ITexPcx9XbdH78UbCAZtirfW1I+pHq+
4T5ZzE0LK3MznmfLU/3tA3JDcDGAQI7SXqGOvHMeeD23kiW/jCoyhnhUQiNshIbj
FN2RfABO3lASHF9tqa5QZwLLffzch5lyKOe0wOOVqRzjAACpEkcw/70CLcmiiSLl
7gLTBiuFoPrvxt09VX2BGJoMoIAI8Y2O1iS2Xdv+7kcra5EjnFn54jeJzDJ2vCpg
vgK/hRP1SyjaPGWkuMdK2TG/q1DsbbtgoEwy1NzFsJ7WIU3n8FEueSrNhB9IUvk0
nRU4XjkpEw==
=p6eN
-----END PGP MESSAGE-----

--=-SS7K3W6vwN1PO8Q4J5EpCg==--

--=-1hQmuiWvF8ISxBVeLWnprg==
Content-Type: multipart/signed; boundary=""=-0sFZ7mtzSwXM4jw8kueeKA=="";
	protocol=""application/pgp-signature""; micalg=pgp-sha256

--=-0sFZ7mtzSwXM4jw8kueeKA==
Content-Type: text/plain

Another MIME text part, this was only signed!
--=-0sFZ7mtzSwXM4jw8kueeKA==
Content-Type: application/pgp-signature; name=signature.pgp
Content-Disposition: attachment; filename=signature.pgp
Content-Transfer-Encoding: 7bit

-----BEGIN PGP MESSAGE-----
Version: 

owJ4nAEfAuD9iQIcBAEBCAAGBQJbJh3iAAoJECVtX0VJiSVbIksP/A7gaFbTWVb5
sbQQqHy5MErh+ZQisB8pwHIonPdlttEK4Q+yE9rw4DVtWszO9XUhJO2xzvGX6Qqw
fkDU0M7ybKG0gMMEhwgfRc9dcSaB8MQyii5wWS9YNTXqNUmoSRS0cP6ka0h65wxh
ISQRnounQcXOIhZZywy3+YY4ZOVbv7uAmgLbBQmYTZa28wPySRlSF6U5p7h0nlJw
4Lf3+4FEtBTCjHPk6OXWpAv9cHWN7BatnoQJjFcGPks/DLy0X3UqJSrHKqOsBSa8
7Lqh/SrGBvOdkdP0Br5SNWKNGAeXrKNte3HgBNY68/IX3hAAPVd3ib2MY3oycyN0
ldzNmtVTU9ECgmz05+v35pU6unpePpKzewa2cO5koL2wnOOACOJoXisMRcXf1K/3
2/d0BP0rDGRJkc09NzLLIlBadU6iTCSo7ReQSZL9KcRCBNHkPrbLJbH3bYJBkdf+
A740RbRRY8CXswWArmAs6hg3b8ZkB8L+F2vTv3p1C311DONsugh9w1LHn9ozG+dC
E4A/zQk7zlyk0SFGm3vFfbdZf9BaL3qzMAzJzBagjO1mACHFGGYvfOcVWEcqIHMM
2U+5ACj3GjHAmGHVS3YiguF1YmLQEeImtLyW3lALrvez/bPkgQEKSXeWB6iLMT2j
MVZXtK9qsFIMQ7XHrMpTFHYewwFj/2wlqL0BXQ==
=IPqN
-----END PGP MESSAGE-----

--=-0sFZ7mtzSwXM4jw8kueeKA==--

--=-1hQmuiWvF8ISxBVeLWnprg==
Content-Type: multipart/mixed; boundary=""=-h2rd7/XUKjoEz8p8LHfVZQ==""

--=-h2rd7/XUKjoEz8p8LHfVZQ==
Content-Type: multipart/signed; boundary=""=-svINFQvFWbaDHisdxgmtFQ=="";
	protocol=""application/pgp-signature""; micalg=pgp-sha256

--=-svINFQvFWbaDHisdxgmtFQ==
Content-Type: application/x-hyperlambda

foo:bar
  Hello:World
  this-was:cryptographically signed!
--=-svINFQvFWbaDHisdxgmtFQ==
Content-Type: application/pgp-signature; name=signature.pgp
Content-Disposition: attachment; filename=signature.pgp
Content-Transfer-Encoding: 7bit

-----BEGIN PGP MESSAGE-----
Version: 

owJ4nAEfAuD9iQIcBAEBCAAGBQJbJh3iAAoJECVtX0VJiSVbBiEQAIWuIvQMkoa9
eMA6z0S0b5YPDtOcHK624fGh075tYz1AmXFHuND5h+2W+2ZMpoIt1Xd3+pbEaxWY
XQ/oCCb0lhkAFQGSKJzi9CNpblWECCt9E87QuSbBUGLJMmFpUUb6exWyB6uWDoMf
I3/cSTWrgbMwrnYBlsZWw3WxH1eUyID8zvPMpveIWgHoNV1REoH5/V6gNcGOEUMi
W1kE1mp4pjzAaN29S14MaqLSzM/Dz+fe/mq+tMytph1kahggs3rZSVqlvN6Io5/U
99Cg4vm38hEDbSGVjyoiNIal0VCOh/G0aHhbd/BrDR9NF7gDwuFPA0jvuxVTXssc
aPdGU4oUvMQ53eV8AFqIEyqfZVDgHNimZ8lbv/UNBqm9Et763UniZC2Xdx9iMUW7
msA3UHTYUYpr/N0uamw69jnzprf3tx00FuyV5Pn8TGkQ/rQiFgYF8QtZwi2LJYc5
wYb0bRSY5h97ynvXvJC/KbXc0e56KTDcRs6xL2/xpnHd7mmlQ8jXJhCY+TTBgIwF
uPQhafvzgAg7ZzWEeuevP7A5vEh0HZMHn0Aa8OYiMSbdVZI/d42jlfIFB8S5+DSG
+4YI4jyt4H1gN0g5RKcL2GAyOkftNySaQINb/047RoyeGvl9tpKXL7bxACZ75kXF
ud+23IkOaLNgTy0LTG2yM2pDhKHLFURVliEFCg==
=pIAf
-----END PGP MESSAGE-----

--=-svINFQvFWbaDHisdxgmtFQ==--

--=-h2rd7/XUKjoEz8p8LHfVZQ==
Content-Type: text/plain

Jo dude! This was not signed, nor encrypted!
--=-h2rd7/XUKjoEz8p8LHfVZQ==--

--=-1hQmuiWvF8ISxBVeLWnprg==--
"
p5.mime.parse
  multipart:mixed
    multipart:encrypted
      signature
        thomas@gaiasoul.com:bool:true
          fingerprint:E43BB8C3A3B9E8DE980051A3256D5F454989255B
      text:plain
        content:Some MIME text part, this was encrypted and signed!
    multipart:signed
      signature
        thomas@gaiasoul.com:bool:true
          fingerprint:E43BB8C3A3B9E8DE980051A3256D5F454989255B
      text:plain
        content:Another MIME text part, this was only signed!
      application:pgp-signature
        Content-Disposition:attachment; filename=signature.pgp
        Content-Transfer-Encoding:7bit
        content:blob:...long blob...
    multipart:mixed
      multipart:signed
        signature
          thomas@gaiasoul.com:bool:true
            fingerprint:E43BB8C3A3B9E8DE980051A3256D5F454989255B
        application:x-hyperlambda
          content
            foo:bar
              Hello:World
              this-was:cryptographically signed!
        application:pgp-signature
          Content-Disposition:attachment; filename=signature.pgp
          Content-Transfer-Encoding:7bit
          content:blob:...long blob...
      text:plain
        content:Jo dude! This was not signed, nor encrypted!

One detail to notice in the above result, is how when you sign or encrypt a MIME entity, the entity is wrapped inside a “multipart:signed”, or “multipart:encrypted” wrapper entity. This is as expected, and necessary to correctly create your MIME envelop. To understand this process, and structure, feel free to run the following through Hypereval.

p5.mime.create
  multipart:mixed
    text:plain
      encrypt
      content:Some MIME text part, this was encrypted!
    text:plain
      sign
      content:Another MIME text part, this was only signed!
p5.mime.parse:x:/@p5.mime.create/0?value

… which of course results in the following …

p5.mime.create
  result:@"Content-Type: multipart/mixed; boundary=""=-DOjAG3V6uxgcYDTlvGj87Q==""

--=-DOjAG3V6uxgcYDTlvGj87Q==
Content-Type: multipart/encrypted; boundary=""=-vpiloBWqr+yJgOppg3UftA=="";
	protocol=""application/pgp-encrypted""

--=-vpiloBWqr+yJgOppg3UftA==
Content-Type: application/pgp-encrypted
Content-Disposition: attachment
Content-Transfer-Encoding: 7bit

Version: 1

--=-vpiloBWqr+yJgOppg3UftA==
Content-Type: application/octet-stream
Content-Disposition: attachment

-----BEGIN PGP MESSAGE-----
Version: 

hQIMAyVtX0VJiSVblah, blah, blah ...
=rNM+
-----END PGP MESSAGE-----

--=-vpiloBWqr+yJgOppg3UftA==--

--=-DOjAG3V6uxgcYDTlvGj87Q==
Content-Type: multipart/signed; boundary=""=-li3Lt5cIlPNcSHm3d0s1eQ=="";
	protocol=""application/pgp-signature""; micalg=pgp-sha256

--=-li3Lt5cIlPNcSHm3d0s1eQ==
Content-Type: text/plain

Another MIME text part, this was only signed!
--=-li3Lt5cIlPNcSHm3d0s1eQ==
Content-Type: application/pgp-signature; name=signature.pgp
Content-Disposition: attachment; filename=signature.pgp
Content-Transfer-Encoding: 7bit

-----BEGIN PGP MESSAGE-----
Version: 

owJ4nAEfAuD9iQblah, blah, blah ...
=ifqy
-----END PGP MESSAGE-----

--=-li3Lt5cIlPNcSHm3d0s1eQ==--

--=-DOjAG3V6uxgcYDTlvGj87Q==--
"
p5.mime.parse
  multipart:mixed
    multipart:encrypted
      text:plain
        content:Some MIME text part, this was encrypted!
    multipart:signed
      signature
        thomas@gaiasoul.com:bool:true
          fingerprint:E43BB8C3A3B9E8DE980051A3256D5F454989255B
      text:plain
        content:Another MIME text part, this was only signed!
      application:pgp-signature
        Content-Disposition:attachment; filename=signature.pgp
        Content-Transfer-Encoding:7bit
        content:blob:... some long blob ...

And since the signing key’s fingerprint is returned as a MIME envelope is being parsed, you can easily do a lookup into (for instance) a database of “trustees”, to see if the MIME envelope was signed by an endpoint you trust or not. This allows you to for instance transfer code over a web service, and execute that code at the web service endpoint, assuming the receiver trusts your PGP key’s fingerprint. However, that is the subject of another article.

MIME headers

Just like HTTP, MIME can also supply “headers”. These are pieces of “meta data” that somehow describes the envelope and entities. Basically, the way you’d supply a MIME header, is by adding it as a simple argument to your MIME entity. Below is an example.

p5.mime.create
  text:plain
    Howdy:World
    content:Foo bar content!
p5.mime.parse:x:/-/0?value

Which results in the following.

p5.mime.create
  result:@"Content-Type: text/plain
Howdy: World

Foo bar content!"
p5.mime.parse
  text:plain
    Howdy:World
    content:Foo bar content!

The idea with a MIME header, and how it is recognised as such by P5, is that it is expected to have at least one CAPITAL letter. Since all the standard MIME headers obeys by this rule, this allows you to add any MIME headers you want to – Including all the standard headers. If you’d like to supply a “Content-Transfer-Encoding” for instance, this can be accomplished with the following.

p5.mime.create
  text:plain
    Content-Transfer-Encoding:base64
    content:Foo bar content!

… resulting in the following …

p5.mime.create
  result:@"Content-Type: text/plain
Content-Transfer-Encoding: base64

Rm9vIGJhciBjb250ZW50IQ==
"

Since MimeKit and Phosphorus Five automatically handles all the default MIME headers, this allows you to use any standard MIME features you want to – In addition to providing any type of additional “meta data” to your MIME entities.

Parsing MIME envelopes directly from disc

The same way we could create a MIME envelope, and persist it to disc in one of our above examples – We can (of course) also parse a MIME entity directly from disc. For instance, in one of our above examples, we created a “foo.mime” file. To parse this file is as simple as follows.

p5.mime.load:/foo.mime

… which results in the following …

p5.mime.load
  text:plain
    content:This is your MIME content!

Of course, you can parametrise your [p5.mime.load] event the same way you can parametrise your [p5.mime.parse] events.

Attachments

The MIME standard has support for attachments. You probably knows this as “email attachments”. However, there is nothing preventing you from using it in any other types of scenarios you want to. However, before we look at file attachments, realise that instead of providing a [content] argument when creating your MIME envelopes, you can optionally instead supply a [filename] argument, pointing to a file on disc. Below is an example.

p5.mime.create
  text:plain
    filename:/startup.hl

Notice, instead of providing inline [content], we declared the content as a [filename] argument. You can of course create as many MIME entities you wish, each pointing to a different [filename].

The above code will create a MIME envelope with a “Content-Disposition” MIME header containing the content of your “startup.hl” file. Its result will resemble the following.

p5.mime.create
  result:@"Content-Type: text/plain
Content-Disposition: attachment; filename=startup.hl

/*
 * Initialises Phosphorus Five.
 *
 * This is being done by evaluating all ""startup.hl"" files, for all modules found
 * in the system.
 */

...blah, blah, blah - The rest of the file ...
"

In fact, unless you explicitly add a [Content-Disposition] argument to your above [p5.mime.create] MIME entity, Phosphorus Five will automatically add one resembling your file, allowing the receiving end automatically deduct that the MIME entity is an attachment. When you parse the MIME envelope you can choose to explicitly save attachments to a folder of your choosing. Below is an example.

p5.mime.create
  text:plain
    filename:/startup.hl
p5.mime.parse:x:/-/0?value
  attachment-folder:~/temp/

The above will not load any attachments into memory, but rather directly persist them to your user’s “temp” folder. This of course preserves memory, allowing you to transfer large files, and/or many files, without exhausting your server’s memory.

Notice, to avoid having filenames crash, and overwrite pre-existing files, the actual physical filename will be “randomised”. This is an automatic feature in Phosphorus Five, making the file name (by default) resemble for instance “597a148fd1984f61be9bddb3c96c4d76-startup.hl”. However, since the [p5.mime.parse] Active Event will return the filename as follows …

p5.mime.parse
  text:plain
    Content-Disposition:attachment; filename=startup.hl
    filename:startup.hl
      prefix:597a148fd1984f61be9bddb3c96c4d76-
      folder:/users/root/temp/

… you can easily deduct the actual physical filename, and its original name, as supplied when the MIME entity was created. To load the file persisted above, you can use for instance the following code.

p5.mime.create
  text:plain
    filename:/startup.hl
p5.mime.parse:x:/-/0?value
  attachment-folder:~/temp/
load-file:~/temp/{0}{1}
  :x:/@p5.mime.parse/0/*/filename/*/prefix?value
  :x:/@p5.mime.parse/0/*/filename?value

If you don’t want the Phosphorus Five’s MIME parser to “randomise” your filenames, you can add [attachments-use-prefix] and set its value to boolean “false”. Below is an example.

p5.mime.create
  text:plain
    filename:/startup.hl
p5.mime.parse:x:/-/0?value
  attachment-folder:~/temp/
  attachments-use-prefix:false

… which will result in the following …

p5.mime.create
  result:@"Content-Type: text/plain
Content-Disposition: attachment; filename=startup.hl

/*
 * Initialises Phosphorus Five.
 *
 * blah, blah, blah - Rest of file ...
 */
"
p5.mime.parse
  text:plain
    Content-Disposition:attachment; filename=startup.hl
    filename:startup.hl
      folder:/users/root/temp/

AND a file in your user’s “temp” folder named *only* “startup.hl”.

About PGP

Open PGP is a cryptography standard, allowing you to (among other things) encrypt data. It is based upon “public key cryptography” or “RSA”. You can watch a 3 minutes long video about it below.

Basically, its idea is that you can encrypt something with a public key. For then to only have the owner of the private key being able to decrypt it.

Its other part is “cryptographically signing” data. When you cryptographically sign something, you use your private key to sign it, and then anybody having access to your public key, can easily verify that the data originates from the guy having the private key – AND that it has not been tampered with since it was created.

When data is encrypted, it is also compressed (think “zipped”). This process significantly reduces the size of your data, in addition to that it becomes impossible for an adversary to see what data is being transferred.

These features combined allows you to use an inheritingly insecure communication channel (HTTP or the “World Wide Web”) to send cryptographically secured data, having a guarantee of that the data originated from who you thought it originated from, has not been read by anybody else but its recipient, and has not been tampered with by anyone since the sender created the data – In addition to that it significantly reduces the size of the data transferred, and such reduces bandwidth consumption when sending your data.

Since PGP is *integral* to Phosphorus Five, and also *blistering fast*, this allows you to do some pretty amazingly cool things with it. Phosphorus Five as of version 8.4, also have no way to export or see the private PGP key, which is by design, and hence your server has now all of a sudden the closest you can come to a 100% perfectly secured communication channel to communicate with other servers, and or encrypt data. Notice, there are no guarantees when it comes to security, but assuming there are no security holes in Phosphorus Five, or the software it relies upon, this literally makes it impossible for an adversary, including heavy duty guys such as the CIA, NSA and FSB to actually see the content transferred between your server, and whomever you choose to communicate with.

Also notice that if you want to create a backup of your server’s private and public PGP key, you can do so by finding your “.gnupg” folder beneath your Apache’s HTML folder, where you can find your PGP key inside of it. You can do such a thing using e.g. SSH. However, to minimise the risk and consequences of any (God forbids) holes in your apps, and/or Phosphorus Five – I have consciously *explicitly removed the ability to export your private PGP keys* in the 8.4 release.

With that in mind, let’s look at some use cases …

Cryptographically secured backups

Creating a really secure backup is difficult. First of all, how do you avoid allowing an adversary to see the content of your backup as you download it, or store it on some medium? Secondly, how do you know nobody have tampered with your backups, and injected some malicious piece of code or data into it, that could compromise your server as you’re restoring your backup? Well, PGP to the rescue.

p5.mime.save:~/temp/backup.mime
  application:x-hyperlambda
    encrypt
    sign
    filename:/startup.hl
micro.download.file:~/temp/backup.mime

The above code will create a backup of your “startup.hl” file, encrypt it, cryptographically sign it, and download it to your client. Meaning even if an adversary can see what is being transferred over the wire, or somehow have access to your backup – He’ll still have no idea what it contains, and the resulting backup file will also be significantly compressed. As you restore your backup, you can check its cryptographic signature, and verify that nobody have tampered with your backup. In fact, since the file is both cryptographically signed, and encrypted, this literally allows you to store your backups in a public cloud somewhere, where anyone can access it – WITHOUT anybody being able to *read* it (or tamper with it). Just remember to verify the signature as you try to restore your backup. I’ll probably at some point create convenience events wrapping this in a future release of Phosphorus Five for the record.

Cryptographically secured web services

Although this is a fairly advanced concept, I’ll go through in another article later, I still want to talk a little bit about it. The idea however, is that if you couple PGP cryptography with lambda web services, which I have written about earlier here. You can all of a sudden, highly securely, allow for execution of Hyperlambda supplied by a client to your server, and such allow only trusted clients to execute Hyperlambda in your web service endpoint. The idea is that you have some sort of list of trusted clients, which are really nothing but fingerprints for PGP keys. And only if the MIME envelope containing the Hyperlambda has been cryptographically signed by one of the clients you happen to trust, you actually go ahead and execute the code transferred.

In fact, this is such a useful scenario, I will create automatic wrappers around it, probably for the 8.4 release of Phosphorus Five.

Securely transferring multiple files

Since MIME allows you to include multiple files in one envelope, and if encrypted, also significantly compress your files – This allows you to transfer hundreds, and actually thousands of files in one simple invocation from one client to some server. The files you transfer can of course be a mixture of text files, binary files, etc – As you see fit …

I will create examples of how to do this in a later article …

Securely storing data on your server

Sometimes a piece of information is so volatile in nature, and contains extremely sensitive content, to such an extent that you need to encrypt it, and/or cryptographically sign it too. In fact, Phosphorus Five contains a very good example of such data; Its “auth.hl” file. This file contains “user settings”, in addition to passwords (in hashed/Blowfish/bcrypt format though), usernames, and other pieces of extremely sensitive information. An adversary having access to this file, can do a lot of damage, since the file might potentially contain things such as your user’s POP3 and SMTP usernames and passwords for instance. PGP to the rescue again!

If you open this file in for instance Hyper IDE, you’ll see that the file is in fact encrypted. This prevents an adversary having access to the physical file to see its content, securing your users’ data, even though a breach of your server should somehow occur.

You are of course free to encrypt any files you create locally in your Phosphorus Five installation as you see fit. With any PGP key you want, allowing your users to for instance create their own private PGP keys, never storing the GnuPG password for these keys, asking them to supply the password when decrypting the files – Eliminating the server’s “root” account from being able to read the files’ content.

AJAX upload forms

If you take a look at the [micro.widgets.upload-button], you’ll see that internally it is actually using the MIME Active Events from Phosphorus Five. This is because as I create the Ajax request on the client with JavaScript, I transfer it as a MIME message to the server, for then to parse it on the server using the MIME events from Phosphorus Five. This allows me to easily use any “advanced browser” features, while using the blistering fast MIME features of Phosphorus Five and Hyperlambda to handle the requests on the server. In addition, due to how MimeKit is being consumed internally in Phosphorus Five, it allows me to upload files in the GIGABYTE size, without exhausting the server’s memory.

MIME is more that just email

I hope I now have proven that MIME is definitely interesting for more than just emails, and in fact highly useful for a lot of different scenarios. Ranging from browser to client communication, backups, and securely PGP encrypted web services …

MIME is truly quite Magical in fact … 😉

Implementing Blowfish password storage

When it comes to security, no amounts of additional layers of security can be redundant. Even though the passwords for Phosphorus Five is already ridiculously secure, due to being PGP encrypted in a file outside of the file system accessible for the system, it doesn’t hurt to implement “slow hashing” – Even though I don’t trust it alone. Simply since it’s first of all a moving target, implying you’ll constantly have to add more workload for your own server to be secure. Secondly because when your server and your adversary’s server are 15 orders of magnitude apart in processing power, making sure your hashing algorithm is secure if it’s only based upon “slow hashing”, simply due to the nature of math, would require so much time to execute locally on your server, that it would require minutes for your users to simply be able to log in.

Regardless, I have now added Blowfish and bcrypt “slow hashing” storage of passwords in the upcoming release of Phosphorus Five. The amount of workload required to hash your passwords are set to 10 by default, but can easily be modified in your web.config. In addition each user now has his own unique salt which is used during hashing, which of course further eliminates an adversary’s ability to crack your passwords, since even two similar passwords in your password file, will still be hashed differently, due to having different salts.

In addition I have included parts of the client’s fingerprint when creating a persistent cookie on disc, if the user chooses to “Remember me” on a specific client. This further reduces the potential for credential “cookie theft”, where an adversary can pick your your cookie somehow, and use it to gain access to the system. It does however invalidate your persistent cookie if you upgrade your browser, but I think this is a small price to pay for the additional security it creates. Needless to say, but this also implies that the persistent credential cookie sent back to the client, is also not the same as the hashed password stored in your authentication file. Since the credential “ticket” sent to the client is hashed, from the Blowfish hashing result, and before it’s hashed it adds the client’s fingerprint (UserAgent and supported languages), and since it’s hashed using SHA256, which has an entropy of 2 to the power of 256, which equals 1.15e+77, which is almost the same number as the number of elementary particles in the observable universe – This makes it as far as I can see redundant to use “slow hashing” on cookie verification, and hence only when the user explicitly needs to log in, there will be a roughly 1-2 second delay while logging in, to perform the slow hashing.

Check out the code here. Using bcrypt was surprisingly easy I must confess. Literally two lines of code after having added the bcrypt package. Really nice work by the developers of bcrypt I must confess 🙂

This means that in the upcoming release, at least when it comes to authentication, Phosphorus Five literally has the same amount of security features that heavy duty things such as Linux and FreeBSD has …

… in fact, arguably better. But don’t tell Linus that … 😉

Software development simplicity

The by far greatest challenge when creating an application, is how to reduce complexity. This is especially true for web apps, since they by the very definition require you to use much more “moving parts”, such as JavaScript, CSS, HTML – In addition to some server-side programming language. Entire libraries have been written about the importance of simplicity in software design, yet these are basically from a practical point of view useless when creating web apps. Simple questions such as “how to I store my passwords for my users” can easily require weeks, if not months of development. Resulting in that before you can even start looking at your domain problem, you’ve spent a year trying to solve “commodity problems”.

Phosphorus Five attempts to solve these problems for you, such that you can start out creating YOUR app from the first second of coding. For instance, how to create a modal window is answered with a handful of lines of code in Phosphorus Five.

micro.widgets.modal
  widgets
    h3
      innerValue:Header of modal window
    // Whatever HTML widgets you want to include additionally

Ask yourself the following question for instance; “How can I (securely) invoke code on my server in an Ajax request, as the user is clicking a button, and how do I create that button?” – Well, that’s 5 lines of code in Phosphorus Five my good friend …

create-widget
  element:button
  innerValue:Click me!
  onclick
    // Do something on your server here ...

It doesn’t take a rocket scientist to understand the advantages of the above compared to hand rolling your own XHR Ajax requests, using JSON to return data from the server, implementing a method in for instance C# that evaluates a piece of code on the server-side, etc, etc, etc. The equivalent jQuery/C#/JavaScript solution for the above piece of code, would easily become 50-100 lines of code, in at least two different programming languages.

Let’s start out with the most difficult task from the beginning of this article, which was as follows; “How do I securely store my users’ passwords?” – Well, in Phosphorus Five that’s ZERO lines of code! Because it’s simply there! Phosphorus Five has a pre-built authentication/authorisation module, allowing you to easily create, edit, and delete users in your system, with some 3-5 lines of code. Below is an example.

p5.auth.users.create:some-username
  password:some-password
  role:some-role

And when the above event has done its job, you have a new user, securely stored in Phosphorus Five’s encrypted users file. This file is encrypted with a PGP key pair, that’s securely stored in GnuPG format, outside of the file system available for your web app. And even if an adversary were to somehow magically be able to decrypt it, which should be impossible might I add – The passwords for your users are still stored as server-side salted hashed values. And the (default) password regime in Phosphorus Five denies a user to create a password that can easily be brute forced attacked with a Rainbow/Dictionary attack, since it requires the password to be at least 25 characters long, hence resulting in that an adversary will need a computer the size of our galaxy to perform a Dictionary attack on it.

If the above paragraph didn’t make any sense for you, relax my friend. The average website’s security is arguably at the point where a teenager with some script kiddy’s skills can literally access your “everything”. With Phosphorus Five, even the CIA or the NSA would highly likely not be able to access *ANYTHING* from your web apps’ sensitive data. Hence, you don’t need to spend months learning how to securely hash your passwords, or securely encrypt your password file, or securely prevent access to your passwords from the application level, which easily might slip for you if you allow for a simple SQL injection attack at any level of your application. With Phosphorus Five you start out with 95% of your job already DONE!

Creating web apps is about reducing complexity. Phosphorus Five is *DEAD FREAKIN’ SIMPLE*!!

And it’s free 🙂

Download Phosphorus Five here

Access control in Phosphorus Five

I have just now significantly refactored authorisation or access control in Phosphorus Five, as you can see from its code. In addition, I’ve also removed a couple of “anomalies”, which arguably were bugs in its code – Some quite severe too may I add. Hence, I wanted to write up about how access to an object is granted or denied in Phosphorus Five, hopefully allowing you to more easily create your own access objects, granting or denying access to specific parts of your Phosphorus Five installation.

First things first, an access object in Phosphorus Five determines whether or not one or more roles have access to some part of your app. The role is the name of the access object’s root node, and whether or not it grants access or denies access to that role to the part in question, is determined according to its name, which can end with either “.deny” or “.allow”. Secondly access objects are “cascading”. What I mean by that, is that they obey to similar rules as a CSS selector. For instance, if I deny access to the path “/foo/” to some role, then unless explicitly overridden in another access object, that same role will be denied access to also “/foo/bar/”.

In addition you can create access objects that are referencing all non-root roles, by creating an access object with a role name of “*”. This implies that the access object is for all roles in the system, except the “root” role, which has access to everything always, and cannot be restricted in any ways. In addition each access object have a “type”. The type declaration of my access objects allows me (or you) to extend the access system, by creating your own types of access objects. By default though, there are the following access objects in Phosphorus Five.

  • p5.module – Determines if access to module is given or not
  • p5.io.read-file – Determines read access to files or folders on disc
  • p5.io.write-file – Determines write access to files or folders on disc
  • p5.hyper-core – Used in Hyper Core to determine access
  • p5.system.platform.execute-file – Used to determine if user has shell execute access to a file on disc

All of the above types are expected to have one of “.allow” or “.deny” after their names. If I wanted to grant access to the “foo” user to write to the files within the folder “/foo/bar/” for instance, I could create an access object resembling the following.

foo
  p5.io.write-file.allow:/foo/bar/

The above would allow all users belonging to the “foo” role to write to all files beneath “/foo/bar/”. Though it presents us with a dilemma, which is that it also allows the user delete the folder or rename it. This might not necessarily be what you want, so you can further restrict this operation, by adding another (parametrised) access object to your list of access objects.

foo
  p5.io.write-file.allow:/foo/bar/
foo
  p5.io.write-file.deny:/foo/bar/
    exact:true

Notice the [exact] parts above. Since a “deny” object always have precedence when two access objects have the same path and role name, if the user tries to rename or delete the “/foo/bar/” folder itself, the last access object from above will have precedence, and hence prevent the user from deleting or renaming the folder itself. However, since the last access object from above has an [exact] argument, it will only match the specified path, if it is exactly “/foo/bar/”. Hence, in our above example we first allows the user to write to everything inside of the folder “/foo/bar/”, for then to deny him to change the “/foo/bar/” folder itself. This gives the “foo” user complete control over everything inside of the “/foo/bar/” folder, but not the folder itself. An access object can be parametrised with the following arguments.

  • [exact] – Requires an exact match
  • [file-type] – A list of pipe separated file extensions, such as e.g. “hl|md|js”
  • [folder] – Requires the path to end with a “/” for the access object to be a match

This gives you an enormous flexibility, allowing you to for instance allow the user to only write to JavaScript and HTML files, restricting write access to all other files in the same folder. Or for instance allowing the user to write to all files inside a folder, but not to create, delete or rename folders. Etc, etc, etc. Below is an example of granting the “designer” role write, create and delete access to HTML, CSS and PNG files for instance inside your “/foo/bar/” folder.

designer
  p5.io.write-file.allow:/foo/bar/
    file-type:html|css|png

If you in addition want to allow the designer role to create folders too, you can accomplish that with the following.

designer
  p5.io.write-file.allow:/foo/bar/
    file-type:html|css|png
designer
  p5.io.write-file.allow:/foo/bar/
    folder:true
designer
  p5.io.write-file.deny:/foo/bar/
    exact:true

The above allows the “designer” role to create, delete, or rename JavaScript files, HTML files and PNG files inside of the “/foo/bar/” folder. It also allows him to create, delete, or rename existing folders inside of the “/foo/bar/” folder, but it prevents him from editing or deleting the actual “/foo/bar/” folder itself. By using the “*” role, you can also give all user access to write to files in some specific folder, and then afterwards restrict one or more roles. The following code allows everybody except the “guest” account to write to HTML files inside of your “/foo/bar/” folder.

*
  p5.io.write-file.allow:/foo/bar/
    file-type:html
guest
  p5.io.write-file.deny:/foo/bar/

The above logic will work since an explicitly named access object (our “guest” object from above), will always have precedence over an “*” object. Since all IO operations will check to see if the user has access to the file according to the access object list, this creates a fairly secure way to grant or deny users access to parts of your Phosphorus Five installation. You can also create your own types of access objects, extending the authorisation features of Phosphorus Five with your own logic – However, that is the subject of another article later …

Password entropy

Ask yourself the following question, which of these two passwords are more easily hacked?

  1. zXHq2$&#
  2. This Is A Password With Some Random Words

The answer might surprise you. If we assume that the user can create a password consisting of 8 characters and each character can be one of 26 capital letters, 26 small letters, 10 different numbers, and a total of 10 different special characters, we have a total amount of different possible combinations of 26+26+10+10 ==> 72 different characters. 72 to the power of 8 ==> 7.2e+14, becomes a number of 7,200,000,000,000,000 different combinations.

The English language contains roughly 150,000 different words. This implies that even assuming every single word in our above phrase starts out with a capital letter, this becomes a total of 150,000 to the power of 8 for a sentence with 8 words. The result of that becomes 2.5e+41. So in fact, that last password from above, is 27 orders of magnitudes more difficult to crack. This is a 1 with 27 zeros behind it!

The last password from above hence is 1,000,000,000,000,000,000,000,000,000 times more difficult to crack!

This implies that if an adversary needs 1 year to crack your 8 character password by brute force, he’ll need 1,000,000,000,000,000,000,000,000,000 years to crack your 8 words password!

Creating passwords with few characters, resembling “rubbish” such as our first example from above, actually provides false security. Simply since to a computer, trying out every single combination of 8 different random “gibberish characters”, is in fact a quite simple task. In addition, to a human being, the last password example from above, is probably hundreds of times easier to remember, than the first password, implying the user doesn’t even need to write down his password to remember it.

Last year the developer who “invented” the above “gibberish” password regime actually publicly put forth his regrets, because 8 random characters, simply doesn’t provide enough “entropy”, for a safe and secure password regime. Entropy is what we measure password strength in. In Phosphorus Five’s upcoming release, one of the things I have changed, is its (default“password regime”. Instead of requiring the user to type in at least on number, one capital letter, one small letter, and one special character, and at least 8 characters in size – I have simply removed all restrictions, except requiring the password to be at least 25 characters long. Allowing you to for instance use a password such as our second example from above. This allows you to use a password that is 1e+27 times more difficult to crack. In addition it allows you to use UNICODE characters, allowing you to create your passwords as Chinese sentences, Russian sentences, or (my native tongue) Norwegian sentences. Literally making it mathematically *impossible* to “crack” your password with brute force.

Now I am a Norwegian native, extending my 150,000 English base line with some additional 70,000 words (the vocabulary of Norwegian). In addition I know some Spanish, some few words of Greek, Italian, French, Arabic, Persian, etc. Extending my base vocabulary with an additional 100,000 per language, since an adversary unless he knows me in person, wouldn’t know what words I know in any of these languages. This becomes a base number of 150,000 for English, plus 100,000 for French, plus 120,000 for Spanish, 70,000 for Norwegian, 100,000 for Greek. Let’s round it of to 750,000. Then comes the fact of that I can start every word with a Capital letter, or not, start only the first word with a capital letter, or not, split words with “_”, “-“, or ” “, etc – Making my “base line” increase by at least 10x, which equals a base line of some 7.5 million. This becomes 1.0e+55, which equals the following number of combinations.

10,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000

The number of elementary particles in the known observable Universe is 1.0e+80. This makes the job of trying to brute force a password with 8 words comparable to naming every single elementary particle in the observable universe! Still, I could easily remember my passwords, such as the following illustrates.

Hasta-La-Vista-I-Morra-Skal-Jeg-Get-Rich

The above would in fact be a very simple password for my brain to remember, and it’s got **9** words, in three different languages, including one dialectic word … 😉

If you’re a system developer, and you’re about to create a password regime for your users, forcing your users to create “gibberish” passwords is actually counterproductive, and creates false security. The best security is in fact to (almost) entirely drop your password restrictions. Math has already scientifically proven this to be a fact!

The additional bonus of course becomes that it makes an entire subject of security obsolete; Per user based server side salts! Since it increases the entropy of your passwords to the point where a Rainbow/Dictionary attack, even having physical access to your password file, would require a computer larger than our observable universe to simply calculate all possible combinations, including a hard drive a 100 trillion times larger than a Galaxy!

Do you trust your RNG?

RNG translates into Random Number Generator, and is at the heart of cryptography. If an adversary can somehow predict your RNG’s output, he can effectively “guess” your encryption keys. There are real valid reasons for why you shouldn’t trust your RNG, depending upon your “paranoia level”. For the average user storing his TODO list encrypted on the web, this has probably few if any implications. However, for a highly paranoid organisation or individual, history have shown us that you probably shouldn’t trust your RNG. Creating truly random numbers without some sort of organic input, is by the very definition of the task literally impossible.

Some developers have proposed suggestions to solve this. All of the best and most paranoid implementations adds some sort of “organic input” to the mix. This can be having the user take a photo that he uses to seed his RNG implementation, listen to static noise over for instance a modem, or read some random bytes from your hard disc. Simply put because a computer cannot create truly random numbers without some sort of organic input.

The way I solve this in Phosphorus Five, is by allowing the user to create an “organic seed” during installation. This seed is cryptographically stored with a private PGP key, which is created by seeding the RNG with the salt the user provides. Below is a screenshot of how this looks like in the UI.

The salt the user applies above, is something he can provide for himself, and this is used to add to the existing entropy of the salting of the RNG from BouncyCastle, before the PGP key is created, that is used to cryptographically secured store the salt. This allows me to later easily create any true RNG number in the system, even if it should be proven in the future that the RNG implementation of BouncyCastle has weaknesses.

By default I use just a cryptographically secure random number, not bothering the user to even ask for a manual salt though, since this could arguably be considered “nuclear rocket security”, and would for the average John Doe be like hunting down a sparrow with a battleship. However, all in all, a pretty rock solid security implementation I’d say, adding that tiny little difference into the mix. So no, you shouldn’t trust your RNG. History has proven that this is probably not wise, at least unless you somehow organically seed it before you start extracting random numbers from it to create cryptography keys.

Only the paranoid survives – CEO of Intel

JavaScript, the Assembly Language of your Generation

Back when I was a kid, somewhere around the times when the Dinosaurs roamed the Earth, most programmers would solve their real problems using Assembly programming. Sure you could create some minor things in BASIC for instance, but all “serious” problems resorted to Assembly. Today we’re in a similar shift, where extremely high level abstractions are not yet (apparently) powerful enough to solve our “real problems”, so we’re left with coding in JavaScript. However, have no doubts, this is an anomaly.

If you attend a job interview 5 years from now, and you explain the guy interviewing you that you’re a skilled JavaScript developer, you’ll be laughed out of the office, the same way a skilled Assembly programmer today is. The reasons are summed up by Paul Graham.

A high level abstraction will always outperform a low level abstraction in productivity

The above lesson was something Graham learned in his venture ViaWeb. And guess what, an employer doesn’t care if you can create a rich web app in 20 days, if somebody else can create it in 5 days, solving his needs. An employer doesn’t care if you’re able to fluently speak Clojure, as if it was your native tongue, if some 5th generation system developer is able to create an Ajax web form 5 times as fast as you.

Even today an extremely skilled Assembly programmer can easily outperform a C++ developer, and implement something that’s at least 25% faster. However, those 25% are basically the difference between 0.00001 and 0.0000075 seconds. Do you really believe the end user is going to notice it …?

What the end user will notice though, is if you’ve spent 10 people working for 10 months, or 2 people working for 2 months to solve the problem. This will from the customer’s point of view literally translate into thousands of dollars, that the software vendor needs to earn, to make the project break even. In addition, your employer will definitely notice it if your competitor is able to deliver a product into production 8 months before you do.

Guess what, JavaScript is an extremely low level abstraction, and if you can avoid it, and use a higher level abstraction – You SHOULD!

NIST, “bcrypt”, Slow Hashing and Elliptic Curve

So, I am in this debate over at Reddit about whether or not I should encrypt my password file, or instead use bcrypt and “slow hashing”. I really didn’t want to go here, but since the argument has started exclusively evolving around “security best practices from NIST”, in addition to bcrypt, which is what NIST recommends developers to use to “secure their passwords” – I feel that I am left with no other choice but defend my view. Which unfortunately will look ugly for NIST.

NIST is an American institution. It’s an acronym, and it means “National Institute of Standards and Technology”. One of its purposes is to propose security standards and best practices for software developers. One of the things NIST has previously standardised is the usage of Elliptic Curve RNGs. RNG translates into “Random Number Generator”. In cryptography having cryptographically secure random numbers is imperative, since without a truly random number, you cannot create encryption keys that are secure. Implying if an adversary can somehow “predict” the output of your RNG, he can accurately re-create your private encryption key.

When NIST standardised the usage of Elliptic Curve RNG, they said that you “should” use two specific numbers, which really was up to the developer to provide himself, but NIST gave their advice on which numbers to put in. Several years passed, and some security expert asked himself the following question about this practice; WUT …?

After some time, a lot of math, and I am assuming a couple of later nights – This expert was able to prove that whoever knew the “distance” between these two numbers, would be able to predict all possible random numbers generated by the algorithm. The security expert even went as far as referring to this as a “backdoor”, and NIST had to apologise, and changed their standards, realising they had been literally taken with their pants down in these matters.

Then Edward Snowden came out and literally showed proof of that the NSA and the CIA had been for years trying to “infiltrate and bribe” standardisation organisations, to create backdoors into standards, which allowed them to access encrypted information. This (obviously) to a large extent explained why Elliptic Curve had been tampered with, though few were willing to say it out loud.

Today NIST has another set of “best practices”. These are practices for how to securely store your passwords, and it’s based upon “slow hashing”. NIST have even proposed a specific library to use for performing this task, and they’ve got hundreds of pages of documentation to show developers why they should choose this path. The problem is that their proposed solution the way I see it, is based upon “raw computational power”. And guess what …

If it boils down to “raw computational power”, it doesn’t take a rocket scientist to understand who’ll “win” here, does it …?

Competing with “raw computational power” against an adversary such as the NSA, CIA, FSB or Chinese intelligence – OR some mafia organisation for that matter, that have access to a botnet with a million computers in their possession – Is the equivalent of having a midget trying to beat Mike Tyson in a boxing match.

Now a midget can in theory beat Mike Tyson. However, not in a “fair fight”. If you gave the midget some sort of advantage, that Mike Tyson did not have, then for a David to give a Goliath a “whopping”, is actually quite easy. We can for instance arm the midget with a baseball bat? Or maybe a tazer? At which point all of a sudden Mike Tyson would be the guy in trouble.

PGP cryptography is that “baseball bat”. Instead of “slow hashing” your passwords, relying upon pure muscle to be able to find your passwords, you can instead simply encrypt your passwords – At which point of course the above datacenter would be left in the dark, and need a million year to figure out your passwords, even WITH physical access to your password file.

There is a saying that goes like the following; “Fool me once, shame on you. Fool me twice, shame on me”. NIST does not have your best interests in mind when they create their “security standards”. Believing they do, would be silly. They’re an American government institution, and just like the CIA, NSA, FBI, “whatever”, they want you to voluntarily hand over all of your data, and your customers’ data too. If they can “trick” you into believing that you’re actually secure as you do this, then they have created an excuse for you possibly, for becoming your customers’ Judas, such that you can’t be pointed at in a court of law for espionage. However, guess what! Just because somebody can’t prove you were the Judas, doesn’t mean you weren’t. Of course, not everybody knows these facts about NIST, which is why I am writing what I am writing here …

However, if you implement “bcrypt” just because NIST told you, you’re an idiot. At least after having checked out the history of Elliptic Curve and NIST’s recommendations here, and/or read what Edward Snowden has to say about these standards.

If I were to ever waste an hour reading what NIST told me where “best practices”, it would in fact be to figure out what NOT to do. NIST is, and have been, for a very long time, simply a branch of the CIA/NSA – And their recommendations are explicitly created in such a way that they shall have access to your data, and your customers’ data. And as they open up backdoors into your data for themselves, they open up backdoors to your data to Chinese intelligence, Russian intelligence, and probably also a couple of intercontinental mafia organisations too in the process.

If you still believe that “bcrypt is secure” after having read this article, then I am sorry to confess, that my best security recommendation to your boss, and your customers, are literally to CHASE YOU OUT OF THEIR BUILDING WITH A STICK!!