Software development simplicity

The by far greatest challenge when creating an application, is how to reduce complexity. This is especially true for web apps, since they by the very definition require you to use much more “moving parts”, such as JavaScript, CSS, HTML – In addition to some server-side programming language. Entire libraries have been written about the importance of simplicity in software design, yet these are basically from a practical point of view useless when creating web apps. Simple questions such as “how to I store my passwords for my users” can easily require weeks, if not months of development. Resulting in that before you can even start looking at your domain problem, you’ve spent a year trying to solve “commodity problems”.

Phosphorus Five attempts to solve these problems for you, such that you can start out creating YOUR app from the first second of coding. For instance, how to create a modal window is answered with a handful of lines of code in Phosphorus Five.

micro.widgets.modal
  widgets
    h3
      innerValue:Header of modal window
    // Whatever HTML widgets you want to include additionally

Ask yourself the following question for instance; “How can I (securely) invoke code on my server in an Ajax request, as the user is clicking a button, and how do I create that button?” – Well, that’s 5 lines of code in Phosphorus Five my good friend …

create-widget
  element:button
  innerValue:Click me!
  onclick
    // Do something on your server here ...

It doesn’t take a rocket scientist to understand the advantages of the above compared to hand rolling your own XHR Ajax requests, using JSON to return data from the server, implementing a method in for instance C# that evaluates a piece of code on the server-side, etc, etc, etc. The equivalent jQuery/C#/JavaScript solution for the above piece of code, would easily become 50-100 lines of code, in at least two different programming languages.

Let’s start out with the most difficult task from the beginning of this article, which was as follows; “How do I securely store my users’ passwords?” – Well, in Phosphorus Five that’s ZERO lines of code! Because it’s simply there! Phosphorus Five has a pre-built authentication/authorisation module, allowing you to easily create, edit, and delete users in your system, with some 3-5 lines of code. Below is an example.

p5.auth.users.create:some-username
  password:some-password
  role:some-role

And when the above event has done its job, you have a new user, securely stored in Phosphorus Five’s encrypted users file. This file is encrypted with a PGP key pair, that’s securely stored in GnuPG format, outside of the file system available for your web app. And even if an adversary were to somehow magically be able to decrypt it, which should be impossible might I add – The passwords for your users are still stored as server-side salted hashed values. And the (default) password regime in Phosphorus Five denies a user to create a password that can easily be brute forced attacked with a Rainbow/Dictionary attack, since it requires the password to be at least 25 characters long, hence resulting in that an adversary will need a computer the size of our galaxy to perform a Dictionary attack on it.

If the above paragraph didn’t make any sense for you, relax my friend. The average website’s security is arguably at the point where a teenager with some script kiddy’s skills can literally access your “everything”. With Phosphorus Five, even the CIA or the NSA would highly likely not be able to access *ANYTHING* from your web apps’ sensitive data. Hence, you don’t need to spend months learning how to securely hash your passwords, or securely encrypt your password file, or securely prevent access to your passwords from the application level, which easily might slip for you if you allow for a simple SQL injection attack at any level of your application. With Phosphorus Five you start out with 95% of your job already DONE!

Creating web apps is about reducing complexity. Phosphorus Five is *DEAD FREAKIN’ SIMPLE*!!

And it’s free 🙂

Download Phosphorus Five here

Access control in Phosphorus Five

I have just now significantly refactored authorisation or access control in Phosphorus Five, as you can see from its code. In addition, I’ve also removed a couple of “anomalies”, which arguably were bugs in its code – Some quite severe too may I add. Hence, I wanted to write up about how access to an object is granted or denied in Phosphorus Five, hopefully allowing you to more easily create your own access objects, granting or denying access to specific parts of your Phosphorus Five installation.

First things first, an access object in Phosphorus Five determines whether or not one or more roles have access to some part of your app. The role is the name of the access object’s root node, and whether or not it grants access or denies access to that role to the part in question, is determined according to its name, which can end with either “.deny” or “.allow”. Secondly access objects are “cascading”. What I mean by that, is that they obey to similar rules as a CSS selector. For instance, if I deny access to the path “/foo/” to some role, then unless explicitly overridden in another access object, that same role will be denied access to also “/foo/bar/”.

In addition you can create access objects that are referencing all non-root roles, by creating an access object with a role name of “*”. This implies that the access object is for all roles in the system, except the “root” role, which has access to everything always, and cannot be restricted in any ways. In addition each access object have a “type”. The type declaration of my access objects allows me (or you) to extend the access system, by creating your own types of access objects. By default though, there are the following access objects in Phosphorus Five.

  • p5.module – Determines if access to module is given or not
  • p5.io.read-file – Determines read access to files or folders on disc
  • p5.io.write-file – Determines write access to files or folders on disc
  • p5.hyper-core – Used in Hyper Core to determine access
  • p5.system.platform.execute-file – Used to determine if user has shell execute access to a file on disc

All of the above types are expected to have one of “.allow” or “.deny” after their names. If I wanted to grant access to the “foo” user to write to the files within the folder “/foo/bar/” for instance, I could create an access object resembling the following.

foo
  p5.io.write-file.allow:/foo/bar/

The above would allow all users belonging to the “foo” role to write to all files beneath “/foo/bar/”. Though it presents us with a dilemma, which is that it also allows the user delete the folder or rename it. This might not necessarily be what you want, so you can further restrict this operation, by adding another (parametrised) access object to your list of access objects.

foo
  p5.io.write-file.allow:/foo/bar/
foo
  p5.io.write-file.deny:/foo/bar/
    exact:true

Notice the [exact] parts above. Since a “deny” object always have precedence when two access objects have the same path and role name, if the user tries to rename or delete the “/foo/bar/” folder itself, the last access object from above will have precedence, and hence prevent the user from deleting or renaming the folder itself. However, since the last access object from above has an [exact] argument, it will only match the specified path, if it is exactly “/foo/bar/”. Hence, in our above example we first allows the user to write to everything inside of the folder “/foo/bar/”, for then to deny him to change the “/foo/bar/” folder itself. This gives the “foo” user complete control over everything inside of the “/foo/bar/” folder, but not the folder itself. An access object can be parametrised with the following arguments.

  • [exact] – Requires an exact match
  • [file-type] – A list of pipe separated file extensions, such as e.g. “hl|md|js”
  • [folder] – Requires the path to end with a “/” for the access object to be a match

This gives you an enormous flexibility, allowing you to for instance allow the user to only write to JavaScript and HTML files, restricting write access to all other files in the same folder. Or for instance allowing the user to write to all files inside a folder, but not to create, delete or rename folders. Etc, etc, etc. Below is an example of granting the “designer” role write, create and delete access to HTML, CSS and PNG files for instance inside your “/foo/bar/” folder.

designer
  p5.io.write-file.allow:/foo/bar/
    file-type:html|css|png

If you in addition want to allow the designer role to create folders too, you can accomplish that with the following.

designer
  p5.io.write-file.allow:/foo/bar/
    file-type:html|css|png
designer
  p5.io.write-file.allow:/foo/bar/
    folder:true
designer
  p5.io.write-file.deny:/foo/bar/
    exact:true

The above allows the “designer” role to create, delete, or rename JavaScript files, HTML files and PNG files inside of the “/foo/bar/” folder. It also allows him to create, delete, or rename existing folders inside of the “/foo/bar/” folder, but it prevents him from editing or deleting the actual “/foo/bar/” folder itself. By using the “*” role, you can also give all user access to write to files in some specific folder, and then afterwards restrict one or more roles. The following code allows everybody except the “guest” account to write to HTML files inside of your “/foo/bar/” folder.

*
  p5.io.write-file.allow:/foo/bar/
    file-type:html
guest
  p5.io.write-file.deny:/foo/bar/

The above logic will work since an explicitly named access object (our “guest” object from above), will always have precedence over an “*” object. Since all IO operations will check to see if the user has access to the file according to the access object list, this creates a fairly secure way to grant or deny users access to parts of your Phosphorus Five installation. You can also create your own types of access objects, extending the authorisation features of Phosphorus Five with your own logic – However, that is the subject of another article later …

Password entropy

Ask yourself the following question, which of these two passwords are more easily hacked?

  1. zXHq2$&#
  2. This Is A Password With Some Random Words

The answer might surprise you. If we assume that the user can create a password consisting of 8 characters and each character can be one of 26 capital letters, 26 small letters, 10 different numbers, and a total of 10 different special characters, we have a total amount of different possible combinations of 26+26+10+10 ==> 72 different characters. 72 to the power of 8 ==> 7.2e+14, becomes a number of 7,200,000,000,000,000 different combinations.

The English language contains roughly 150,000 different words. This implies that even assuming every single word in our above phrase starts out with a capital letter, this becomes a total of 150,000 to the power of 8 for a sentence with 8 words. The result of that becomes 2.5e+41. So in fact, that last password from above, is 27 orders of magnitudes more difficult to crack. This is a 1 with 27 zeros behind it!

The last password from above hence is 1,000,000,000,000,000,000,000,000,000 times more difficult to crack!

This implies that if an adversary needs 1 year to crack your 8 character password by brute force, he’ll need 1,000,000,000,000,000,000,000,000,000 years to crack your 8 words password!

Creating passwords with few characters, resembling “rubbish” such as our first example from above, actually provides false security. Simply since to a computer, trying out every single combination of 8 different random “gibberish characters”, is in fact a quite simple task. In addition, to a human being, the last password example from above, is probably hundreds of times easier to remember, than the first password, implying the user doesn’t even need to write down his password to remember it.

Last year the developer who “invented” the above “gibberish” password regime actually publicly put forth his regrets, because 8 random characters, simply doesn’t provide enough “entropy”, for a safe and secure password regime. Entropy is what we measure password strength in. In Phosphorus Five’s upcoming release, one of the things I have changed, is its (default“password regime”. Instead of requiring the user to type in at least on number, one capital letter, one small letter, and one special character, and at least 8 characters in size – I have simply removed all restrictions, except requiring the password to be at least 25 characters long. Allowing you to for instance use a password such as our second example from above. This allows you to use a password that is 1e+27 times more difficult to crack. In addition it allows you to use UNICODE characters, allowing you to create your passwords as Chinese sentences, Russian sentences, or (my native tongue) Norwegian sentences. Literally making it mathematically *impossible* to “crack” your password with brute force.

Now I am a Norwegian native, extending my 150,000 English base line with some additional 70,000 words (the vocabulary of Norwegian). In addition I know some Spanish, some few words of Greek, Italian, French, Arabic, Persian, etc. Extending my base vocabulary with an additional 100,000 per language, since an adversary unless he knows me in person, wouldn’t know what words I know in any of these languages. This becomes a base number of 150,000 for English, plus 100,000 for French, plus 120,000 for Spanish, 70,000 for Norwegian, 100,000 for Greek. Let’s round it of to 750,000. Then comes the fact of that I can start every word with a Capital letter, or not, start only the first word with a capital letter, or not, split words with “_”, “-“, or ” “, etc – Making my “base line” increase by at least 10x, which equals a base line of some 7.5 million. This becomes 1.0e+55, which equals the following number of combinations.

10,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000

The number of elementary particles in the known observable Universe is 1.0e+80. This makes the job of trying to brute force a password with 8 words comparable to naming every single elementary particle in the observable universe! Still, I could easily remember my passwords, such as the following illustrates.

Hasta-La-Vista-I-Morra-Skal-Jeg-Get-Rich

The above would in fact be a very simple password for my brain to remember, and it’s got **9** words, in three different languages, including one dialectic word … 😉

If you’re a system developer, and you’re about to create a password regime for your users, forcing your users to create “gibberish” passwords is actually counterproductive, and creates false security. The best security is in fact to (almost) entirely drop your password restrictions. Math has already scientifically proven this to be a fact!

The additional bonus of course becomes that it makes an entire subject of security obsolete; Per user based server side salts! Since it increases the entropy of your passwords to the point where a Rainbow/Dictionary attack, even having physical access to your password file, would require a computer larger than our observable universe to simply calculate all possible combinations, including a hard drive a 100 trillion times larger than a Galaxy!

Do you trust your RNG?

RNG translates into Random Number Generator, and is at the heart of cryptography. If an adversary can somehow predict your RNG’s output, he can effectively “guess” your encryption keys. There are real valid reasons for why you shouldn’t trust your RNG, depending upon your “paranoia level”. For the average user storing his TODO list encrypted on the web, this has probably few if any implications. However, for a highly paranoid organisation or individual, history have shown us that you probably shouldn’t trust your RNG. Creating truly random numbers without some sort of organic input, is by the very definition of the task literally impossible.

Some developers have proposed suggestions to solve this. All of the best and most paranoid implementations adds some sort of “organic input” to the mix. This can be having the user take a photo that he uses to seed his RNG implementation, listen to static noise over for instance a modem, or read some random bytes from your hard disc. Simply put because a computer cannot create truly random numbers without some sort of organic input.

The way I solve this in Phosphorus Five, is by allowing the user to create an “organic seed” during installation. This seed is cryptographically stored with a private PGP key, which is created by seeding the RNG with the salt the user provides. Below is a screenshot of how this looks like in the UI.

The salt the user applies above, is something he can provide for himself, and this is used to add to the existing entropy of the salting of the RNG from BouncyCastle, before the PGP key is created, that is used to cryptographically secured store the salt. This allows me to later easily create any true RNG number in the system, even if it should be proven in the future that the RNG implementation of BouncyCastle has weaknesses.

By default I use just a cryptographically secure random number, not bothering the user to even ask for a manual salt though, since this could arguably be considered “nuclear rocket security”, and would for the average John Doe be like hunting down a sparrow with a battleship. However, all in all, a pretty rock solid security implementation I’d say, adding that tiny little difference into the mix. So no, you shouldn’t trust your RNG. History has proven that this is probably not wise, at least unless you somehow organically seed it before you start extracting random numbers from it to create cryptography keys.

Only the paranoid survives – CEO of Intel

JavaScript, the Assembly Language of your Generation

Back when I was a kid, somewhere around the times when the Dinosaurs roamed the Earth, most programmers would solve their real problems using Assembly programming. Sure you could create some minor things in BASIC for instance, but all “serious” problems resorted to Assembly. Today we’re in a similar shift, where extremely high level abstractions are not yet (apparently) powerful enough to solve our “real problems”, so we’re left with coding in JavaScript. However, have no doubts, this is an anomaly.

If you attend a job interview 5 years from now, and you explain the guy interviewing you that you’re a skilled JavaScript developer, you’ll be laughed out of the office, the same way a skilled Assembly programmer today is. The reasons are summed up by Paul Graham.

A high level abstraction will always outperform a low level abstraction in productivity

The above lesson was something Graham learned in his venture ViaWeb. And guess what, an employer doesn’t care if you can create a rich web app in 20 days, if somebody else can create it in 5 days, solving his needs. An employer doesn’t care if you’re able to fluently speak Clojure, as if it was your native tongue, if some 5th generation system developer is able to create an Ajax web form 5 times as fast as you.

Even today an extremely skilled Assembly programmer can easily outperform a C++ developer, and implement something that’s at least 25% faster. However, those 25% are basically the difference between 0.00001 and 0.0000075 seconds. Do you really believe the end user is going to notice it …?

What the end user will notice though, is if you’ve spent 10 people working for 10 months, or 2 people working for 2 months to solve the problem. This will from the customer’s point of view literally translate into thousands of dollars, that the software vendor needs to earn, to make the project break even. In addition, your employer will definitely notice it if your competitor is able to deliver a product into production 8 months before you do.

Guess what, JavaScript is an extremely low level abstraction, and if you can avoid it, and use a higher level abstraction – You SHOULD!

NIST, “bcrypt”, Slow Hashing and Elliptic Curve

So, I am in this debate over at Reddit about whether or not I should encrypt my password file, or instead use bcrypt and “slow hashing”. I really didn’t want to go here, but since the argument has started exclusively evolving around “security best practices from NIST”, in addition to bcrypt, which is what NIST recommends developers to use to “secure their passwords” – I feel that I am left with no other choice but defend my view. Which unfortunately will look ugly for NIST.

NIST is an American institution. It’s an acronym, and it means “National Institute of Standards and Technology”. One of its purposes is to propose security standards and best practices for software developers. One of the things NIST has previously standardised is the usage of Elliptic Curve RNGs. RNG translates into “Random Number Generator”. In cryptography having cryptographically secure random numbers is imperative, since without a truly random number, you cannot create encryption keys that are secure. Implying if an adversary can somehow “predict” the output of your RNG, he can accurately re-create your private encryption key.

When NIST standardised the usage of Elliptic Curve RNG, they said that you “should” use two specific numbers, which really was up to the developer to provide himself, but NIST gave their advice on which numbers to put in. Several years passed, and some security expert asked himself the following question about this practice; WUT …?

After some time, a lot of math, and I am assuming a couple of later nights – This expert was able to prove that whoever knew the “distance” between these two numbers, would be able to predict all possible random numbers generated by the algorithm. The security expert even went as far as referring to this as a “backdoor”, and NIST had to apologise, and changed their standards, realising they had been literally taken with their pants down in these matters.

Then Edward Snowden came out and literally showed proof of that the NSA and the CIA had been for years trying to “infiltrate and bribe” standardisation organisations, to create backdoors into standards, which allowed them to access encrypted information. This (obviously) to a large extent explained why Elliptic Curve had been tampered with, though few were willing to say it out loud.

Today NIST has another set of “best practices”. These are practices for how to securely store your passwords, and it’s based upon “slow hashing”. NIST have even proposed a specific library to use for performing this task, and they’ve got hundreds of pages of documentation to show developers why they should choose this path. The problem is that their proposed solution the way I see it, is based upon “raw computational power”. And guess what …

If it boils down to “raw computational power”, it doesn’t take a rocket scientist to understand who’ll “win” here, does it …?

Competing with “raw computational power” against an adversary such as the NSA, CIA, FSB or Chinese intelligence – OR some mafia organisation for that matter, that have access to a botnet with a million computers in their possession – Is the equivalent of having a midget trying to beat Mike Tyson in a boxing match.

Now a midget can in theory beat Mike Tyson. However, not in a “fair fight”. If you gave the midget some sort of advantage, that Mike Tyson did not have, then for a David to give a Goliath a “whopping”, is actually quite easy. We can for instance arm the midget with a baseball bat? Or maybe a tazer? At which point all of a sudden Mike Tyson would be the guy in trouble.

PGP cryptography is that “baseball bat”. Instead of “slow hashing” your passwords, relying upon pure muscle to be able to find your passwords, you can instead simply encrypt your passwords – At which point of course the above datacenter would be left in the dark, and need a million year to figure out your passwords, even WITH physical access to your password file.

There is a saying that goes like the following; “Fool me once, shame on you. Fool me twice, shame on me”. NIST does not have your best interests in mind when they create their “security standards”. Believing they do, would be silly. They’re an American government institution, and just like the CIA, NSA, FBI, “whatever”, they want you to voluntarily hand over all of your data, and your customers’ data too. If they can “trick” you into believing that you’re actually secure as you do this, then they have created an excuse for you possibly, for becoming your customers’ Judas, such that you can’t be pointed at in a court of law for espionage. However, guess what! Just because somebody can’t prove you were the Judas, doesn’t mean you weren’t. Of course, not everybody knows these facts about NIST, which is why I am writing what I am writing here …

However, if you implement “bcrypt” just because NIST told you, you’re an idiot. At least after having checked out the history of Elliptic Curve and NIST’s recommendations here, and/or read what Edward Snowden has to say about these standards.

If I were to ever waste an hour reading what NIST told me where “best practices”, it would in fact be to figure out what NOT to do. NIST is, and have been, for a very long time, simply a branch of the CIA/NSA – And their recommendations are explicitly created in such a way that they shall have access to your data, and your customers’ data. And as they open up backdoors into your data for themselves, they open up backdoors to your data to Chinese intelligence, Russian intelligence, and probably also a couple of intercontinental mafia organisations too in the process.

If you still believe that “bcrypt is secure” after having read this article, then I am sorry to confess, that my best security recommendation to your boss, and your customers, are literally to CHASE YOU OUT OF THEIR BUILDING WITH A STICK!!

Here is my “weakly hashed” password file – Feel free to try to crack it

When it comes to security, there is a lot of dogmatic beliefs out there. For instance, some guy recommended that I hash my passwords thousands of times. This was due to that if my hashing algorithm took one second to execute, a Rainbow/Dictionary attack brute forcing my passwords, for then to perform a lookup towards the hash values of my password file, would simply not work. These are considered “best practice” in our industry, and you can find entire sections at StackOverflow.Com arguing for this approach. In fact, there are even multiple libraries written for this sole purpose.

There are two problems with that approach. Both problems arises from the fact of that Phosphorus Five is implemented in C#. This implies that what’s a “slow hashing function” in C#, can easily be “lubricated” in assembly or C to become blistering fast! The second problem is that each iteration of hashing would require some heap memory, making the garbage collector kick in every n times a user tries to login – Rendering the system for all practical concerns USELESS!

So instead of relying upon “best practices” in regards to this problem, I asked myself what IS the problem. Well, the problem is that if an adversary gains physical access to your password file for some reasons, he can gain access to your passwords. The first time we “fixed” this problem, we fixed it by hashing our passwords, for then to never store our passwords in plain text. Then some jerk came around and figured he could use a Rainbow attack to brute force your password file. This works in such a way that he generates the hash value for every possible combination of characters that could in theory be used as a password. Generating every single hash value, for every single possible combination of characters in the alphabet up to 8 characters in length, requires surprisingly small amount of time, and can actually be done in seconds, with very few resources. Then he could simply take an existing hashed password, find its instance in his “Rainbow database”, and such find your password.

So we started salting our passwords, with a “per user” salt, to make sure even if an adversary manages to crack one of the passwords in your file, he still won’t be able to do a lookup for multiple occurrences in your password file, having the same hash value. In addition, we started “slow hashing”. Slow hashing implies that we hash thousands of times, resulting in that generating the hash for a single password combination, takes at least one second. Implying that creating this “dictionary” of pre-hashed values would require too much CPU time to be of practical usage. First of all, this implies adding a LOT of CPU overhead to your application. Secondly, what is “slow” for your server, is easily within the reach of a teenager with $10,000 to rent a server farm for some few hours, and some small amounts of C/Assembly knowledge. What is slow for your server and C#, is basically peanuts for a million servers running Assembly code. An organisation such as the NSA, CIA or the FSB (**PUN!**) could eat through your “slow hashing” in milliseconds, without even noticing a “blip” on their server farm’s CPU usage …!

So you must assume that the FSB basically knows all of your password. Because this has been “industry best practices” for a decade or so, and hence “all” developers have chosen this path – Including yours … 😉

So I figured that the “best practices” in these regards were arguably broken, and effectively useless. So instead of doing a “slow hash”, I decided to rip up the problem by its roots, and instead storing the password file encrypted. Just to prove hos secure this is, I challenge my readers to figure out my password. Here is my password file …

gnupg-keypair:1EF027678DEB0FCF06266C448C4A8A3647CC8B3B
Content-Type: multipart/encrypted; boundary="=-LLyo/DkZazvC4JmU6M3Qag==";
	protocol="application/pgp-encrypted"

--=-LLyo/DkZazvC4JmU6M3Qag==
Content-Type: application/pgp-encrypted
Content-Disposition: attachment
Content-Transfer-Encoding: 7bit

Version: 1

--=-LLyo/DkZazvC4JmU6M3Qag==
Content-Type: application/octet-stream
Content-Disposition: attachment

-----BEGIN PGP MESSAGE-----
Version: 

hQIMA4xKijZHzIs7AQ//U5kniwVL6jOR1onW5Itr3CZ5BvL3LykxcjMYyVC2Qdjj
RZ4EWZ3W++i4CgTLynwxWz47kq20Mn9RmKbVkXqWsj51bHGAbIi/U6np/STlrWaJ
/9dnDSR9UeaLmpP0h0uVmWrzCx2NmljGSY8ClrCWKN1iNhSlIDGgVNYf2mpps20l
C4bqO+HYiYWWdr9uesPlk+MKJrIgKoGZq9OY3kYp2pS1+w4t2M2/rwP8sx8cWXOk
TV2Bp+NQIIccGEwzRpDiTKtZANI+uQX0c7IzQEYAXm1EfKH636NlCQtGNWYR3Dex
R4flBFUvzSp4eJa3/B+Y3eqs0R55wMubqSXsPzVCp5kMFTv1xSnJbsRsUoPBeNS6
1T92mlHB1Uviy5FOktNJrAuMtw5E8Z7ZD4/0sXIeplo9fKt6XoO3ou7w7CKckxoH
lUIc/OBx8N0dAljYICpR4IObNPb3T3XRJ3m/+Onsf0NBJIrznLUUGtCFE/GFrMRL
4DVU9P0cAWyVAwHzZUgoejfw8g04jmIld0dLttPeq0oBZxiG5T04hBlarivl2Nb9
M31fdg8fmD2C9hDI/W68BT07SaoVkLDoUTA7c56mOtHnjLhYGqULIKpL3nCaWUg8
efj7n6P5b8eavGY+2DgGenrdblWgypk9H61iSuTcaNMk3fGkbAlKcwPXEyus2gvS
tQGtb/sNVJS7PqhY3Lg28PQN2AlSw3DfUsdy4WfyRNDUgIPMHA0C22eRTsNVqyVN
tzO7F/n8FkaYDH2hZstocgcMHnAxUcOzanO/DQF4m49UtfMHcb6Vhhlh2X2zDUok
L8CDiVWC7XNjhjyapiJGA/I3NdMjQWwrhbdvFFH3qiuPGybMnUIjAX+kcPCGY024
ieVj67Lntuh+ILnIDqdrJPXRfAJ9xhUpBNR2/sv5AxDpFWiNm9A=
=fHHa
-----END PGP MESSAGE-----

--=-LLyo/DkZazvC4JmU6M3Qag==--

Now try to figure out my password … 😉

Good Luck!

PS!
This of course implies that you can literally store your password file, as I have done above, as plain text on your blog. Which of course makes it much easier to create backups of your password file, in addition to providing much better security than “slow hashing”.

PPS!
Due to that “slow hashing” have been our industry’s “best practices” for a long time, an estimated guess would be that 99% of all web apps in this world have password implementations, that could easily be hacked by a teenager, with $10,000 to rent a server farm, and some above average C/ASM knowledge, in a couple of hours …

PPPS!
If that makes your paranoid, I happen to know the solution to your problem 😀

Creating Hyperlambda from C#

I’ve just been refactoring and cleaning the code where I access my “auth” file in Phosphorus Five. The “auth” file contains the usernames/passwords and authorisation objects of Phosphorus Five. Since I am invoking Active Events parametrised from within these methods, I thought I’d take some time to explain this, since it provides an excellent use case of how to “create Hyperlambda” from C#.

The first thing you must realise about Hyperlambda, is that it’s simply a node structure. In fact, when your Hyperlambda has been parsed, its result is a simple graph object, encapsulated in the Node class of Phosphorus Five. This class contains lots of helper methods, but can be reduced down to the following pseudo code.

class Node {
    string Name;
    object Value;
    list Children;
}

The Children parts above is a list of Node children.

This trait of Hyperlambda, allows us to create Hyperlambda from C#. This might sound weird, but actually has lots of use cases. For instance, I wanted to encrypt my “auth” file when saving it, and decrypt it when loading it. I could of course have done this manually, by referencing BouncyCastle directly. However, I already have helper Active Events, that I can reuse, making my code much more condense, and allowing me to create more “DRY” code. DRY meaning “Don’t Repeat Yourself”. So what I ended up with was the following.


/*
 * Implementation of saving auth file.
 */
static void SaveAuthFileInternal (ApplicationContext context)
{
    // Getting path.
    string pwdFilePath = GetAuthFilePath (context);

    // Saving file, making sure we encrypt it in the process.
    using (TextWriter writer = new StreamWriter (File.Create (pwdFilePath))) {

        // Retrieving fingerprint from auth file, and removing the fingerprint node, since
        // it's not supposed to be save inside of the ecnrypted MIME part of our auth file's content.
        var fingerprint = _authFileContent ["gnupg-keypair"].UnTie ().Get<string> (context);

        try {

            // Writing fingerprint of PGP key used to encrypt auth file at the top of the file.
            writer.WriteLine (string.Format ("gnupg-keypair:{0}", fingerprint));

            // Encrypting auth file's content.
            var node = new Node ();
            node.Add ("text", "plain").LastChild
                .Add ("content", Utilities.Convert<string> (context, _authFileContent.Children))
                .Add ("encrypt").LastChild
                .Add ("fingerprint", fingerprint);
            context.RaiseEvent ("p5.mime.create", node);

            // Writing encrypted content to stream.
            writer.Write (node ["result"].Get<string> (context));

        } finally {

            // Adding back fingerprint to cached auth file.
            _authFileContent.Insert (0, new Node ("gnupg-keypair", fingerprint));
        }
    }
}

The above code is all the code I need to save my “auth” file, using a PGP key, stored in my GnuPG storage. Of course, if I was to use BouncyCastle directly, the above code would literally explode in size, and probably grow to at least one order of magnitude more in size. Instead I get to reuse my existing MIME Active Events, that allows me to simply supply a PGP key fingerprint, which will lookup my PGP key from my GnuPG storage, create a “text:plain” MIME envelope, encrypt that envelope, before it saves that encrypted envelope to disc. The thing you should pay particularly notice to above, is the parts where I construct my “node” object. The above code’s Hyperlambda equivalent would be the following.

p5.mime.create
  text:plain
    content:my-auth-file-content
    encrypt
      fingerprint:some-GnuPG-fingerprint

Of course, arguably using a MIME envelope to serialise my auth file to disc, arguably creates some overhead. However, this overhead is a small price to pay for the simplified code. In addition it also allows me to easily extend my logic later, to for instance create multipart MIME envelopes, wrapping multiple auth files, encrypted with their own unique PGP keys. In addition to that it allows me to serialise binary data later into my auth file, etc, etc, etc. So the added overhead of constructing a MIME envelope wrapping my auth file, is really quite insignificant compared to its gains. Reading the file from disc and decrypting it, is equally easy.


/*
 * Private implementation of retrieving auth file.
 */
static Node GetAuthFileInternal (ApplicationContext context)
{
    // Checking if we can return cached version.
    if (_authFileContent != null)
        return _authFileContent;

    // Getting path.
    string pwdFilePath = GetAuthFilePath (context);

    // Checking if file exist.
    if (!File.Exists (pwdFilePath)) {
        _authFileContent = new Node ("").Add ("users"); // First time retrieval of "auth" file.
        return _authFileContent;
    }

    // Reads auth file and decrypts it.
    using (TextReader reader = new StreamReader (File.OpenRead (pwdFilePath))) {

        // Retrieving fingerprint for PGP key to use to decrypt file.
        var fingerprintLine = reader.ReadLine ();
        var fingerprint = fingerprintLine.Split (':') [1];

        // Retrieving GnuPG key's password from web.config.
        var confNode = new Node ("", "gpg-server-keypair-password");
        var gnuPgPassword = context.RaiseEvent ("p5.config.get", confNode).FirstChild.Get<string> (context);
        
        // Retrieving the rest of the content of file.
        var fileContent = reader.ReadToEnd ();

        // Decrypting file's content with PGP key referenced at the top of the file.
        var node = new Node ("", fileContent);
        node.Add ("decrypt").LastChild
            .Add ("fingerprint", fingerprint).LastChild
            .Add ("password", gnuPgPassword);
        context.RaiseEvent ("p5.mime.parse", node);

        // Converting Hyperlambda content of file to a node, caching it, and returning it to caller.
        // Making sure we explicitly add the [gnupg-keypair] to the "auth" node first.
        _authFileContent = Utilities.Convert<Node> (context, node.FirstChild ["text"] ["content"].Value);
        _authFileContent.Add ("gnupg-keypair", fingerprint);
        return _authFileContent;
    }
}

The above code will in a similar fashion to its save counterpart, decrypt my file with a handful of lines of code. Probably saving me hundreds of lines of code, compared to if I were to explicitly implement decryption instead. Its Hyperlambda counterpart would resemble the following.

/*
 * Retrieving the password for our PGP key.
 */
p5.config.get:gpg-server-keypair-password
p5.mime.parse:auth-file-content-goes-here
  decrypt
    fingerprint:x:/@p5.config.get/*?value
      password:gnupg-password-goes-here

All in all saving me for hundreds, if not thousands of lines of code, allowing me to keep my code “DRY”, and reuse as much of my existing implementations as possible, without creating references between my two different projects. The last point is crucial. Because my auth file wrappers can be found in the project called “p5.auth”, while my MIME wrappers can be found in “p5.mime”. If you look at these projects in Visual Studio, you will notice that none of these projects have any references to the other, eliminating all the problems that such references might create, such as versioning problems, changing the signatures of my classes methods, etc, etc, etc. Resulting in a solution where I can reuse existing functionality from other projects, without any references between my projects at all!

I think that’s pretty cool! What do you think …?

An independent Security Expert’s Code Review of Phosphorus Five

It’s really quite fascinating what you can get people to do for you for free, if you just “adequately motivate them”, and give them access to your source code. I’ve had several security experts from Reddit over the last couple of days literally scrutinising my code with a microscope, looking for security holes. Especially one guy truly emerged as a champion in this process, Mr. Cifize. Cifize was able to find several weaknesses in Phosphorus Five, all of which are now tightened. If I were to hire people professionally to do what Cifize did for Phosphorus Five for free, it would probably have costed me somewhere between 10,000 and 20,000 dollars. I am of course very grateful to Cifize for what he have done for Phosphorus Five. Thank you Cifize 🙂

Passwords

Although my existing password file was already protected quite well, Cifize pointed out that a brute force rainbow attack, done by an adversary who already had access to the file, could “reverse engineer” its passwords, by brute force. Hence, my existing server-side salting hashing logic of my user’s passwords, could probably need some tightening up. Hence, I followed Cifize’s advice, and significantly tightened the way I store passwords.

The way I chose to do this, was to make sure I encrypt the password file, with a 4096bits RSA PGP key. This key is internally stored on the server encrypted with AES, which makes it even tighter. The password used to release the key from the GnuPG keyring, is stored in web.config. While the private PGP key used to decrypt the password file, is stored in GnuPG. Since GnuPG stores its keys outside of the filesystem that Phosphorus Five has access to, this makes it almost impossible to retrieve the PGP key, even for an adversary with full “root” access (P5 “root” access) to your server.

So an adversary will need literally almost complete access to your server, simply to be able to decrypt your password file. And even at that point, if he should somehow be able to decrypt your passwords, they’re still internally stored as server-side salted hashed values. On could argue that this is close to insanity and pure paranoia in regards to security – But when it comes to security, you should be paranoid! Better to add some 5-6 additional extra layers of security, than one too few … 😉

Additional security fixes

In addition Cifize was able to find a place in the backup methods which could in theory make an adversary able to perform an SQL injection attack. Although this could only occur if an adversary somehow was able to trick a “root” account to import a malicious CSV file, I still chose to fix it, to be on the safe side.

In addition, there were some minor issues with the “install.sh” script, that installs Phosphorus Five on a Linux machine. To make sure I install your server now in adequately paranoia mode, I’ve completely removed all HTTP headers that can positively identify details about the underlaying technology.

Since security is a constant ongoing process though, I would like to encourage all my readers to send me an email, if you should discover a hole. In addition, I have created a “honey pot server” myself, which you are welcome to try to hack. If you want to have a go at trying to hack Phosphorus Five, you can do so here.

Yet again I would like to give my thanks to Cifize, who have proven to be an invaluable asset in this process. Thank you Cifize 🙂

You can download the latest version here

Please hack my server

Some developers are spreading vicious and incorrect rumours about Hyperlambda and Phosphorus Five on the internet. To combat these rumours, I thought I’d prove its security. The way I have chosen to do that, is to create a small page, allowing anonymous access to execute “eval” on my server. My intentions are to prove that developers sometimes mindlessly repeats dogmatic teachings, without understanding the context where they are relevant.

For instance, in Phosphorus Five and Hyperlambda “eval” is perfectly safe, if you use it correctly. This is because Hyperlambda has an “overload” of eval, which allows you to supply a list of legal Active Events to it, preventing the user from using insecure Active Events, that might produce dangerous side effects. This allows you to literally use [eval-whitelist] for some really interesting things, such as for instance creating “lambda web services”, where the client supplies the code to be executed, without compromising security. Of course, those spreading rumours about Phosphorus Five’s insecurity, simply “avoid adding this ‘tiny little detail'” as they are claiming “It’s insecure, it contains ‘eval’ all over the place”. This list of things they happen to exclude as they’re rambling on about Phosphorus Five’s “insecurity” is much longer too, but since this seems to be the “most dangerous thing about Phosphorus Five” – I thought if they can’t hack into it even though I have given them execute “eval” permissions, most other things they are claiming is probably not true either … 😉

You can find the app here. And you can also create HTTP POST requests towards the same URL, and provide your Hyperlambda as the body of your request, at which point the result of your execution will be returned back to your client. Below is the entire code that I used to create this page.

/*
 * Creates a Hyperlambda "eval" page.
 */
p5.web.request.get-method
if:x:/-?value
  =:POST

  /*
   * Web Service invocation.
   * Retrieving body of request, and executing it using [eval-whitelist].
   */
  p5.web.request.get-body
  hyper2lambda:x:/-?value
  eval-whitelist:x:/-
    events
      set
      add
      src
      return
      hypereval.snippets.load
      hypereval.snippets.search
  lambda2hyper:x:/-/*
  p5.web.echo:x:/-?value
  return


/*
 * Creates a default page, with a header and a paragraph.
 */
create-widget
  class:container
  oninit

    /*
     * Including Micro CSS file, serious skin, and fonts.
     */
    micro.css.include

  widgets
    div
      class:row
      widgets
        div
          class:col
          widgets
            h1
              innerValue:Hack my server challenge

            /*
             * CodeMirror instance.
             */
            micro.widgets.codemirror:hyperlambda
              mode:hyperlambda
              auto-focus:true
            div
              class:right
              widgets
                button
                  innerValue:Execute
                  onclick

                    /*
                     * Retrieves code, executes it, and creates a modal window with
                     * the results of the execution.
                     */
                    micro.widgets.codemirror.get-value:hyperlambda
                    hyper2lambda:x:/-/*?value
                    eval-whitelist:x:/-
                      events
                        set
                        add
                        src
                        return
                        hypereval.snippets.load
                        hypereval.snippets.search
                    eval-x:x:/+/*/*/*/*
                    create-widgets
                      micro.widgets.modal
                        widgets
                          pre
                            innerValue:x:/@eval-whitelist

Notice, you also have indirectly access to read from my MySQL database, since I have whitelisted a couple of the Hypereval “snippets” Active Events. If you can break into my server, using a security flaw in Phosphorus Five, I will publicly admit that Phosphorus Five is insecure, and allow you to fill an article at my blog, with whatever content you want to fill it with. I will basically allow you to write a blog at my website, spreading anything you want to inform my users about, related to Phosphorus Five, me, and my person. And I will link to that article with a bold “warning” from the project’s GitHub page, at the top of its README.md file.

I have only one criterion. Obviously I cannot guarantee that Linux, Ubuntu, MySQL, Apache, or any of the other software pieces my box is using are safe – Even though I am pretty confident in that also these projects are pretty safe, considering the amount of security additions Phosphorus Five applies to your Linux box, as it is being installed. However, I will ask of you that you use a security hole in Phosphorus Five, and not a hole in any of its supporting software, and that you prove you did, by handing me a reproducible, which I can use to verify you used a security hole in Phosphorus Five, and not one in Linux or Apache etc …

Good luck! 😉

Epilogue; The next time you hear a mindless dogmatic belief, try to ask yourself 2 questions.

  1. What’s the motive of those putting forth the claims
  2. What is the context of what they are saying

For instance, lots of users will attack Phosphorus Five for using a server-side salt when hashing its passwords. This is completely irrelevant for a system such as Phosphorus Five, since its intention is to be an enterprise web application framework, with probably never more than maximum a 1,000 registered users. This means that the statistical probability of having two passwords collide, is so small, that the added complexity of creating a “per user based salt”, only results in added complexity, arguably reducing its security.

Security is always about “beating the odds”. And to be able to apply security adequately, you need to understand the context and the reasons for why we do things the way we do. Simply “following best practices”, without understanding the reasons why these were created in the first place, actually reduces security – Instead of improving it …

I also want to emphasise that my linux box, has not added a single additional security concern, beyond what the default installation script of Phosphorus Five applies. Still, I am willing to bet my honour on that you won’t be able to penetrate it – At least not due to a security hole in Phosphorus Five!

If you manage to somehow hack it though, you can send your reproducible to thomas@gaiasoul.com, together with whatever text you want to provide to my users to warn them from using Phosphorus Five.