Here is my “weakly hashed” password file – Feel free to try to crack it

When it comes to security, there is a lot of dogmatic beliefs out there. For instance, some guy recommended that I hash my passwords thousands of times. This was due to that if my hashing algorithm took one second to execute, a Rainbow/Dictionary attack brute forcing my passwords, for then to perform a lookup towards the hash values of my password file, would simply not work. These are considered “best practice” in our industry, and you can find entire sections at StackOverflow.Com arguing for this approach. In fact, there are even multiple libraries written for this sole purpose.

There are two problems with that approach. Both problems arises from the fact of that Phosphorus Five is implemented in C#. This implies that what’s a “slow hashing function” in C#, can easily be “lubricated” in assembly or C to become blistering fast! The second problem is that each iteration of hashing would require some heap memory, making the garbage collector kick in every n times a user tries to login – Rendering the system for all practical concerns USELESS!

So instead of relying upon “best practices” in regards to this problem, I asked myself what IS the problem. Well, the problem is that if an adversary gains physical access to your password file for some reasons, he can gain access to your passwords. The first time we “fixed” this problem, we fixed it by hashing our passwords, for then to never store our passwords in plain text. Then some jerk came around and figured he could use a Rainbow attack to brute force your password file. This works in such a way that he generates the hash value for every possible combination of characters that could in theory be used as a password. Generating every single hash value, for every single possible combination of characters in the alphabet up to 8 characters in length, requires surprisingly small amount of time, and can actually be done in seconds, with very few resources. Then he could simply take an existing hashed password, find its instance in his “Rainbow database”, and such find your password.

So we started salting our passwords, with a “per user” salt, to make sure even if an adversary manages to crack one of the passwords in your file, he still won’t be able to do a lookup for multiple occurrences in your password file, having the same hash value. In addition, we started “slow hashing”. Slow hashing implies that we hash thousands of times, resulting in that generating the hash for a single password combination, takes at least one second. Implying that creating this “dictionary” of pre-hashed values would require too much CPU time to be of practical usage. First of all, this implies adding a LOT of CPU overhead to your application. Secondly, what is “slow” for your server, is easily within the reach of a teenager with $10,000 to rent a server farm for some few hours, and some small amounts of C/Assembly knowledge. What is slow for your server and C#, is basically peanuts for a million servers running Assembly code. An organisation such as the NSA, CIA or the FSB (**PUN!**) could eat through your “slow hashing” in milliseconds, without even noticing a “blip” on their server farm’s CPU usage …!

So you must assume that the FSB basically knows all of your password. Because this has been “industry best practices” for a decade or so, and hence “all” developers have chosen this path – Including yours … 😉

So I figured that the “best practices” in these regards were arguably broken, and effectively useless. So instead of doing a “slow hash”, I decided to rip up the problem by its roots, and instead storing the password file encrypted. Just to prove hos secure this is, I challenge my readers to figure out my password. Here is my password file …

gnupg-keypair:1EF027678DEB0FCF06266C448C4A8A3647CC8B3B
Content-Type: multipart/encrypted; boundary="=-LLyo/DkZazvC4JmU6M3Qag==";
	protocol="application/pgp-encrypted"

--=-LLyo/DkZazvC4JmU6M3Qag==
Content-Type: application/pgp-encrypted
Content-Disposition: attachment
Content-Transfer-Encoding: 7bit

Version: 1

--=-LLyo/DkZazvC4JmU6M3Qag==
Content-Type: application/octet-stream
Content-Disposition: attachment

-----BEGIN PGP MESSAGE-----
Version: 

hQIMA4xKijZHzIs7AQ//U5kniwVL6jOR1onW5Itr3CZ5BvL3LykxcjMYyVC2Qdjj
RZ4EWZ3W++i4CgTLynwxWz47kq20Mn9RmKbVkXqWsj51bHGAbIi/U6np/STlrWaJ
/9dnDSR9UeaLmpP0h0uVmWrzCx2NmljGSY8ClrCWKN1iNhSlIDGgVNYf2mpps20l
C4bqO+HYiYWWdr9uesPlk+MKJrIgKoGZq9OY3kYp2pS1+w4t2M2/rwP8sx8cWXOk
TV2Bp+NQIIccGEwzRpDiTKtZANI+uQX0c7IzQEYAXm1EfKH636NlCQtGNWYR3Dex
R4flBFUvzSp4eJa3/B+Y3eqs0R55wMubqSXsPzVCp5kMFTv1xSnJbsRsUoPBeNS6
1T92mlHB1Uviy5FOktNJrAuMtw5E8Z7ZD4/0sXIeplo9fKt6XoO3ou7w7CKckxoH
lUIc/OBx8N0dAljYICpR4IObNPb3T3XRJ3m/+Onsf0NBJIrznLUUGtCFE/GFrMRL
4DVU9P0cAWyVAwHzZUgoejfw8g04jmIld0dLttPeq0oBZxiG5T04hBlarivl2Nb9
M31fdg8fmD2C9hDI/W68BT07SaoVkLDoUTA7c56mOtHnjLhYGqULIKpL3nCaWUg8
efj7n6P5b8eavGY+2DgGenrdblWgypk9H61iSuTcaNMk3fGkbAlKcwPXEyus2gvS
tQGtb/sNVJS7PqhY3Lg28PQN2AlSw3DfUsdy4WfyRNDUgIPMHA0C22eRTsNVqyVN
tzO7F/n8FkaYDH2hZstocgcMHnAxUcOzanO/DQF4m49UtfMHcb6Vhhlh2X2zDUok
L8CDiVWC7XNjhjyapiJGA/I3NdMjQWwrhbdvFFH3qiuPGybMnUIjAX+kcPCGY024
ieVj67Lntuh+ILnIDqdrJPXRfAJ9xhUpBNR2/sv5AxDpFWiNm9A=
=fHHa
-----END PGP MESSAGE-----

--=-LLyo/DkZazvC4JmU6M3Qag==--

Now try to figure out my password … 😉

Good Luck!

PS!
This of course implies that you can literally store your password file, as I have done above, as plain text on your blog. Which of course makes it much easier to create backups of your password file, in addition to providing much better security than “slow hashing”.

PPS!
Due to that “slow hashing” have been our industry’s “best practices” for a long time, an estimated guess would be that 99% of all web apps in this world have password implementations, that could easily be hacked by a teenager, with $10,000 to rent a server farm, and some above average C/ASM knowledge, in a couple of hours …

PPPS!
If that makes your paranoid, I happen to know the solution to your problem 😀

Advertisements

Creating Hyperlambda from C#

I’ve just been refactoring and cleaning the code where I access my “auth” file in Phosphorus Five. The “auth” file contains the usernames/passwords and authorisation objects of Phosphorus Five. Since I am invoking Active Events parametrised from within these methods, I thought I’d take some time to explain this, since it provides an excellent use case of how to “create Hyperlambda” from C#.

The first thing you must realise about Hyperlambda, is that it’s simply a node structure. In fact, when your Hyperlambda has been parsed, its result is a simple graph object, encapsulated in the Node class of Phosphorus Five. This class contains lots of helper methods, but can be reduced down to the following pseudo code.

class Node {
    string Name;
    object Value;
    list Children;
}

The Children parts above is a list of Node children.

This trait of Hyperlambda, allows us to create Hyperlambda from C#. This might sound weird, but actually has lots of use cases. For instance, I wanted to encrypt my “auth” file when saving it, and decrypt it when loading it. I could of course have done this manually, by referencing BouncyCastle directly. However, I already have helper Active Events, that I can reuse, making my code much more condense, and allowing me to create more “DRY” code. DRY meaning “Don’t Repeat Yourself”. So what I ended up with was the following.


/*
 * Implementation of saving auth file.
 */
static void SaveAuthFileInternal (ApplicationContext context)
{
    // Getting path.
    string pwdFilePath = GetAuthFilePath (context);

    // Saving file, making sure we encrypt it in the process.
    using (TextWriter writer = new StreamWriter (File.Create (pwdFilePath))) {

        // Retrieving fingerprint from auth file, and removing the fingerprint node, since
        // it's not supposed to be save inside of the ecnrypted MIME part of our auth file's content.
        var fingerprint = _authFileContent ["gnupg-keypair"].UnTie ().Get<string> (context);

        try {

            // Writing fingerprint of PGP key used to encrypt auth file at the top of the file.
            writer.WriteLine (string.Format ("gnupg-keypair:{0}", fingerprint));

            // Encrypting auth file's content.
            var node = new Node ();
            node.Add ("text", "plain").LastChild
                .Add ("content", Utilities.Convert<string> (context, _authFileContent.Children))
                .Add ("encrypt").LastChild
                .Add ("fingerprint", fingerprint);
            context.RaiseEvent ("p5.mime.create", node);

            // Writing encrypted content to stream.
            writer.Write (node ["result"].Get<string> (context));

        } finally {

            // Adding back fingerprint to cached auth file.
            _authFileContent.Insert (0, new Node ("gnupg-keypair", fingerprint));
        }
    }
}

The above code is all the code I need to save my “auth” file, using a PGP key, stored in my GnuPG storage. Of course, if I was to use BouncyCastle directly, the above code would literally explode in size, and probably grow to at least one order of magnitude more in size. Instead I get to reuse my existing MIME Active Events, that allows me to simply supply a PGP key fingerprint, which will lookup my PGP key from my GnuPG storage, create a “text:plain” MIME envelope, encrypt that envelope, before it saves that encrypted envelope to disc. The thing you should pay particularly notice to above, is the parts where I construct my “node” object. The above code’s Hyperlambda equivalent would be the following.

p5.mime.create
  text:plain
    content:my-auth-file-content
    encrypt
      fingerprint:some-GnuPG-fingerprint

Of course, arguably using a MIME envelope to serialise my auth file to disc, arguably creates some overhead. However, this overhead is a small price to pay for the simplified code. In addition it also allows me to easily extend my logic later, to for instance create multipart MIME envelopes, wrapping multiple auth files, encrypted with their own unique PGP keys. In addition to that it allows me to serialise binary data later into my auth file, etc, etc, etc. So the added overhead of constructing a MIME envelope wrapping my auth file, is really quite insignificant compared to its gains. Reading the file from disc and decrypting it, is equally easy.


/*
 * Private implementation of retrieving auth file.
 */
static Node GetAuthFileInternal (ApplicationContext context)
{
    // Checking if we can return cached version.
    if (_authFileContent != null)
        return _authFileContent;

    // Getting path.
    string pwdFilePath = GetAuthFilePath (context);

    // Checking if file exist.
    if (!File.Exists (pwdFilePath)) {
        _authFileContent = new Node ("").Add ("users"); // First time retrieval of "auth" file.
        return _authFileContent;
    }

    // Reads auth file and decrypts it.
    using (TextReader reader = new StreamReader (File.OpenRead (pwdFilePath))) {

        // Retrieving fingerprint for PGP key to use to decrypt file.
        var fingerprintLine = reader.ReadLine ();
        var fingerprint = fingerprintLine.Split (':') [1];

        // Retrieving GnuPG key's password from web.config.
        var confNode = new Node ("", "gpg-server-keypair-password");
        var gnuPgPassword = context.RaiseEvent ("p5.config.get", confNode).FirstChild.Get<string> (context);
        
        // Retrieving the rest of the content of file.
        var fileContent = reader.ReadToEnd ();

        // Decrypting file's content with PGP key referenced at the top of the file.
        var node = new Node ("", fileContent);
        node.Add ("decrypt").LastChild
            .Add ("fingerprint", fingerprint).LastChild
            .Add ("password", gnuPgPassword);
        context.RaiseEvent ("p5.mime.parse", node);

        // Converting Hyperlambda content of file to a node, caching it, and returning it to caller.
        // Making sure we explicitly add the [gnupg-keypair] to the "auth" node first.
        _authFileContent = Utilities.Convert<Node> (context, node.FirstChild ["text"] ["content"].Value);
        _authFileContent.Add ("gnupg-keypair", fingerprint);
        return _authFileContent;
    }
}

The above code will in a similar fashion to its save counterpart, decrypt my file with a handful of lines of code. Probably saving me hundreds of lines of code, compared to if I were to explicitly implement decryption instead. Its Hyperlambda counterpart would resemble the following.

/*
 * Retrieving the password for our PGP key.
 */
p5.config.get:gpg-server-keypair-password
p5.mime.parse:auth-file-content-goes-here
  decrypt
    fingerprint:x:/@p5.config.get/*?value
      password:gnupg-password-goes-here

All in all saving me for hundreds, if not thousands of lines of code, allowing me to keep my code “DRY”, and reuse as much of my existing implementations as possible, without creating references between my two different projects. The last point is crucial. Because my auth file wrappers can be found in the project called “p5.auth”, while my MIME wrappers can be found in “p5.mime”. If you look at these projects in Visual Studio, you will notice that none of these projects have any references to the other, eliminating all the problems that such references might create, such as versioning problems, changing the signatures of my classes methods, etc, etc, etc. Resulting in a solution where I can reuse existing functionality from other projects, without any references between my projects at all!

I think that’s pretty cool! What do you think …?

An independent Security Expert’s Code Review of Phosphorus Five

It’s really quite fascinating what you can get people to do for you for free, if you just “adequately motivate them”, and give them access to your source code. I’ve had several security experts from Reddit over the last couple of days literally scrutinising my code with a microscope, looking for security holes. Especially one guy truly emerged as a champion in this process, Mr. Cifize. Cifize was able to find several weaknesses in Phosphorus Five, all of which are now tightened. If I were to hire people professionally to do what Cifize did for Phosphorus Five for free, it would probably have costed me somewhere between 10,000 and 20,000 dollars. I am of course very grateful to Cifize for what he have done for Phosphorus Five. Thank you Cifize 🙂

Passwords

Although my existing password file was already protected quite well, Cifize pointed out that a brute force rainbow attack, done by an adversary who already had access to the file, could “reverse engineer” its passwords, by brute force. Hence, my existing server-side salting hashing logic of my user’s passwords, could probably need some tightening up. Hence, I followed Cifize’s advice, and significantly tightened the way I store passwords.

The way I chose to do this, was to make sure I encrypt the password file, with a 4096bits RSA PGP key. This key is internally stored on the server encrypted with AES, which makes it even tighter. The password used to release the key from the GnuPG keyring, is stored in web.config. While the private PGP key used to decrypt the password file, is stored in GnuPG. Since GnuPG stores its keys outside of the filesystem that Phosphorus Five has access to, this makes it almost impossible to retrieve the PGP key, even for an adversary with full “root” access (P5 “root” access) to your server.

So an adversary will need literally almost complete access to your server, simply to be able to decrypt your password file. And even at that point, if he should somehow be able to decrypt your passwords, they’re still internally stored as server-side salted hashed values. On could argue that this is close to insanity and pure paranoia in regards to security – But when it comes to security, you should be paranoid! Better to add some 5-6 additional extra layers of security, than one too few … 😉

Additional security fixes

In addition Cifize was able to find a place in the backup methods which could in theory make an adversary able to perform an SQL injection attack. Although this could only occur if an adversary somehow was able to trick a “root” account to import a malicious CSV file, I still chose to fix it, to be on the safe side.

In addition, there were some minor issues with the “install.sh” script, that installs Phosphorus Five on a Linux machine. To make sure I install your server now in adequately paranoia mode, I’ve completely removed all HTTP headers that can positively identify details about the underlaying technology.

Since security is a constant ongoing process though, I would like to encourage all my readers to send me an email, if you should discover a hole. In addition, I have created a “honey pot server” myself, which you are welcome to try to hack. If you want to have a go at trying to hack Phosphorus Five, you can do so here.

Yet again I would like to give my thanks to Cifize, who have proven to be an invaluable asset in this process. Thank you Cifize 🙂

You can download the latest version here

Please hack my server

Some developers are spreading vicious and incorrect rumours about Hyperlambda and Phosphorus Five on the internet. To combat these rumours, I thought I’d prove its security. The way I have chosen to do that, is to create a small page, allowing anonymous access to execute “eval” on my server. My intentions are to prove that developers sometimes mindlessly repeats dogmatic teachings, without understanding the context where they are relevant.

For instance, in Phosphorus Five and Hyperlambda “eval” is perfectly safe, if you use it correctly. This is because Hyperlambda has an “overload” of eval, which allows you to supply a list of legal Active Events to it, preventing the user from using insecure Active Events, that might produce dangerous side effects. This allows you to literally use [eval-whitelist] for some really interesting things, such as for instance creating “lambda web services”, where the client supplies the code to be executed, without compromising security. Of course, those spreading rumours about Phosphorus Five’s insecurity, simply “avoid adding this ‘tiny little detail'” as they are claiming “It’s insecure, it contains ‘eval’ all over the place”. This list of things they happen to exclude as they’re rambling on about Phosphorus Five’s “insecurity” is much longer too, but since this seems to be the “most dangerous thing about Phosphorus Five” – I thought if they can’t hack into it even though I have given them execute “eval” permissions, most other things they are claiming is probably not true either … 😉

You can find the app here. And you can also create HTTP POST requests towards the same URL, and provide your Hyperlambda as the body of your request, at which point the result of your execution will be returned back to your client. Below is the entire code that I used to create this page.

/*
 * Creates a Hyperlambda "eval" page.
 */
p5.web.request.get-method
if:x:/-?value
  =:POST

  /*
   * Web Service invocation.
   * Retrieving body of request, and executing it using [eval-whitelist].
   */
  p5.web.request.get-body
  hyper2lambda:x:/-?value
  eval-whitelist:x:/-
    events
      set
      add
      src
      return
      hypereval.snippets.load
      hypereval.snippets.search
  lambda2hyper:x:/-/*
  p5.web.echo:x:/-?value
  return


/*
 * Creates a default page, with a header and a paragraph.
 */
create-widget
  class:container
  oninit

    /*
     * Including Micro CSS file, serious skin, and fonts.
     */
    micro.css.include

  widgets
    div
      class:row
      widgets
        div
          class:col
          widgets
            h1
              innerValue:Hack my server challenge

            /*
             * CodeMirror instance.
             */
            micro.widgets.codemirror:hyperlambda
              mode:hyperlambda
              auto-focus:true
            div
              class:right
              widgets
                button
                  innerValue:Execute
                  onclick

                    /*
                     * Retrieves code, executes it, and creates a modal window with
                     * the results of the execution.
                     */
                    micro.widgets.codemirror.get-value:hyperlambda
                    hyper2lambda:x:/-/*?value
                    eval-whitelist:x:/-
                      events
                        set
                        add
                        src
                        return
                        hypereval.snippets.load
                        hypereval.snippets.search
                    eval-x:x:/+/*/*/*/*
                    create-widgets
                      micro.widgets.modal
                        widgets
                          pre
                            innerValue:x:/@eval-whitelist

Notice, you also have indirectly access to read from my MySQL database, since I have whitelisted a couple of the Hypereval “snippets” Active Events. If you can break into my server, using a security flaw in Phosphorus Five, I will publicly admit that Phosphorus Five is insecure, and allow you to fill an article at my blog, with whatever content you want to fill it with. I will basically allow you to write a blog at my website, spreading anything you want to inform my users about, related to Phosphorus Five, me, and my person. And I will link to that article with a bold “warning” from the project’s GitHub page, at the top of its README.md file.

I have only one criterion. Obviously I cannot guarantee that Linux, Ubuntu, MySQL, Apache, or any of the other software pieces my box is using are safe – Even though I am pretty confident in that also these projects are pretty safe, considering the amount of security additions Phosphorus Five applies to your Linux box, as it is being installed. However, I will ask of you that you use a security hole in Phosphorus Five, and not a hole in any of its supporting software, and that you prove you did, by handing me a reproducible, which I can use to verify you used a security hole in Phosphorus Five, and not one in Linux or Apache etc …

Good luck! 😉

Epilogue; The next time you hear a mindless dogmatic belief, try to ask yourself 2 questions.

  1. What’s the motive of those putting forth the claims
  2. What is the context of what they are saying

For instance, lots of users will attack Phosphorus Five for using a server-side salt when hashing its passwords. This is completely irrelevant for a system such as Phosphorus Five, since its intention is to be an enterprise web application framework, with probably never more than maximum a 1,000 registered users. This means that the statistical probability of having two passwords collide, is so small, that the added complexity of creating a “per user based salt”, only results in added complexity, arguably reducing its security.

Security is always about “beating the odds”. And to be able to apply security adequately, you need to understand the context and the reasons for why we do things the way we do. Simply “following best practices”, without understanding the reasons why these were created in the first place, actually reduces security – Instead of improving it …

I also want to emphasise that my linux box, has not added a single additional security concern, beyond what the default installation script of Phosphorus Five applies. Still, I am willing to bet my honour on that you won’t be able to penetrate it – At least not due to a security hole in Phosphorus Five!

If you manage to somehow hack it though, you can send your reproducible to thomas@gaiasoul.com, together with whatever text you want to provide to my users to warn them from using Phosphorus Five.

The future is obsolete

For a couple of hundreds of years, we have lived in an economical society, where our labour possessed value for others. And as new paradigms came and went, such as the agricultural society, the industrial society, service based society, knowledge based society, we always seemed to be able to adapt. That stops in a very near future. Less than a decade from now, there is literally no job what so ever, that a computer or a robot for $1000 cannot do better, more accurately, and less expensive.

When Deep Blue humiliated Gary Kasparov in 1996/97, it took only 15 years before the average pocket calculator could beat the world chess champion, without even making an effort. Today the same research team that created Deep Blue, is creating doctors and physicians. This implies that 15 years from now, your physician and your surgeon will be a pocket calculator. We’re already seeing signs of this, such as self driving cars, automated supermarkets, the drones Amazon are using, etc, etc, etc. Steady but slowly, computers and robots will entirely replace every single task human beings are able to perform today. And I can’t wait!

Hallelujah, humans are obsolete soon!! 😀

If work was so valuable, the rich would keep all the jobs for themselves

Contrary to others, I literally can’t wait. I’m looking forward to this future, rejoicing, realising I can spend more time exercising my hobbies – Which paradoxically includes playing chess. The future is in fact so bright, I’ve gotta wear shades!

If you’re a youngster today, and you’re wondering about what education to take, let me give you my advice. First of all, when it comes to knowledge, skills, and “thinking”, there is not a single place in this world where you cannot be outperformed by the average pocket calculator 15 years from now. So spending your time at universities, reciting endless (and useless) information based facts, or the very art of logic itself, is completely useless. (Logic is Greek for “thinking”) – Hence your PhD 15 years from now, is about as useless as your ability to ride a horse and a carriage is for the average employer today. However, there are places where you can outperform computers, at least for the foreseeable future. This is where emotions are important, and the fact of that you’re actually a human being.

For instance, regardless of how intelligent our computers and robots becomes, and regardless of how perfect they can play the Saxophone, many people will still enjoy seeing an actual person playing the Saxophone as they’re visiting their local pub. Simply because we enjoy surrounding ourselves with talented human beings. The same way most people enjoy talking to an actual waitress as they’re ordering their coffee – If not for any other reasons than making a flirty remark to her as she takes your order.

People still enjoys watching Magnus Carlsen play chess, because of his personality, his body language, and the way he conducts himself in public. In fact, the (actual) world chess championship contest have at most a handful of spectators, since these competitions are being performed between super computers, which are a thousand times better than Magnus Carlsen, and performing their competition in dark rooms, deep inside of mountain halls, with no physical chess board, and no lights – In temperatures below freezing, to make sure their “brains doesn’t boil over”. This is the *actual* world chess championship. But nobody cares, because they find Magnus Carlsen, although significantly inferior to these machines, to be a nice human being, and they enjoy watching him playing chess. We even refer to Magnus as the “world champion”, even though we all know for a fact that the last human world chess champion was in fact Gary Kasparov in 1996.

If you’re basing your future on your cognitive superiority, and your logic, and your ability to “think”, you’re in trouble. A piece of advice, relax, be more human, be nicer, and calm down. Otherwise Deep Blue is going to kick your ass out of your office, while your boss is laughing, thinking “finally I got my revenge over that fucking arrogant asshole”.

Peace out,

“Just another Saxophone player”

And do you want to know a secret? You, yes you – As a system developer that is, is making everybody else in this world obsolete. There seems to be some kind of justice to the fact that somebody actually makes YOU obsolete …

… and most people will probably laugh, and cheer on me, as I do it!

Create web apps without JavaScript, CSS, HTML and C#

One of the things about Phosphorus Five, is that it allows you to create fairly rich web apps, without knowing any JavaScript, HTML, CSS, or C# – Or any other server side backend language for that matter. You can get away with this because most problems you’ll encounter in your web apps are already solved in the framework. For instance, to show a login dialogue is one line of Hyperlambda. To create a datagrid wrapping your MySQL database is 7 lines of declarative code. To create an Ajax tree view that shows the folders on your (server) disc is another 27 lines of code. A modal window, another 5 lines of code. Ask the user to confirm some action, 3 lines of Hyperlambda. Since Hyperlambda “feels” more like YAML than C# or JavaScript, it doesn’t feel like you’re actually coding.

Ask yourself how many of your problems are unique to your app. Chances are most of the things you need to do, have been done by thousands of other developers previously, in thousands of apps. If this wasn’t true, why do we find so many answers to our questions when we’re stuck and we choose to Google our problems? Phosphorus Five takes advantage of that fact, and facilitates for making it extremely easy to create reusable code and components. In fact, when you create your web apps in Phosphorus Five, you don’t create a monolithic app – It’s simply impossible. What you end up creating, is a whole range of reusable components, that you can put into your current app, and your next app. Since I have already created a whole range of different apps with Phosphorus Five and Hyperlambda, ranging from webmail clients to IDEs, there’s a component in Phosphorus Five for most of your needs. This literally allows you to create your app, almost exclusively using pre-built components, with a close to zero code base.

According to rumours I heard once, Visual Studio contains roughly 1.5 million lines of code. Hyper IDE contains 2429 lines of code, if you don’t count the comments and spacing. Visual Studio contains 617 times more code than Hyper IDE. Obviously Visual Studio contains tons of features that Hyper IDE does not. However, most of those features are things you’ll never miss, and things you didn’t even realise was there. In fact, I doubt there’s a single person in this world, including the project manager on the Visual Studio team, that can even mention every single feature in VS. Secondly, Hyper IDE has its own unique traits too, which Visual Studio does not. For instance, the ability to use it from your phone, access it from any terminal, granting write access to users only for specific files, etc, etc, etc. Being roughly 10x faster and more responsive for most tasks of course, also obviously helps. Being a gazillion times easier to extend, is also an extreme advantage. In fact, there’s a name for apps such as Visual Studio, it’s not a very flattering name either. The name we use for such apps is “bloatware”.

When I built Hyper IDE I already had all the main components I needed. This was the reason why I didn’t need more than 2429 lines of code. I had a CodeMirror editor widget, since I had already created one for Hypereval. I had a tree view Ajax widget, to show the files on disc. I had a toolbar widget, which I had already used in many other apps. I had modal windows from before. Etc, etc, etc.

When you create an app in Phosphorus Five, you don’t need to create as much code as you’ll need to create when you use “whatever else”. First of all, because most of your problems, are problems I have already solved. Secondly, the problems that you’ll actually need to solve, is by default possible to solve such that when you have solved them, you can reuse your solution in your next project – Without creating much dependencies between your apps might I add. This allows you to solve at least 80% of your problem, sometimes 100%, without resorting to writing JavaScript, HTML, or CSS. This again allows you to create rich and highly advanced server-side functionality, without having to write code in PHP, C#, or any other server-side programming language for that matter.

Every time you reinvent the wheel, you are stealing from your employer

The above might sound drastic, but you as a developer have a cost. This is often an hourly cost. Implying if you do things you don’t have to do, you are indirectly stealing from your employer. You probably don’t intend to steal – However, your vanity and your “not invented here syndrome”, prevents you from seeing the truth. So you end up spending your employers money, to learn something, that is completely useless – Because you want to become the “best developer in the world”. This is literally stealing!

If you’re an average full stack software developer, ask yourself the following questions; How much CISC x86 assembly code do you know? How does the pipeline in the latest Intel CPU work? What is the L1 cache size of your web server? If you’re an average software developer, you’ll have no idea how to answer those questions. Sure, you can probably easily find the answers by using Google, if you need these answers. But the fact that you are now probably frenetically Googling trying to find the answers to the above questions illustrates my point. Below is a piece of code. If you don’t count its comments and spacing, there’s 27 lines of code in it.

/*
 * Creates a page with a tree widget in it, which displays
 * all the folders on disc.
 */
create-widget
  class:container
  oninit

    /*
     * Including Micro CSS file.
     */
    micro.css.include

  widgets
    div
      class:row
      widgets
        div
          class:col
          widgets
            h1
              innerValue:Tree view widget

            /*
             * This is our treeview widget.
             */
            micro.widgets.tree
              items
                root:/

              /*
               * This one will be invoked when the tree needs items.
               * It will be given an [_item-id] argument.
               *
               * We simply list the folders of the item the tree needs
               * children for here.
               */
              .onexpand
                list-folders:x:/../*/_item-id?value
                for-each:x:/@list-folders/*?name
                  split:x:/@_dp?value
                    =:/
                  add:x:/../*/return/*
                    src:@"{0}:{1}"
                      :x:/@split/0/-?name
                      :x:/@_dp?value
                return
                  items

Here is what it results in …

The only thing you need to know about the above code, is what it does, when to use it, and that it works! It’s secure, it’s extremely efficient on bandwidth, and it shows your folder structure on your disc. The same is true for your CPU. You don’t need to worry about how large its L1 cache is. You only need to know that it works. If you can understand how the above [.onexpand] Hyperlambda work, you can easily change it, to make it show other things, besides your folders.

In fact, if the above wasn’t true, the idea of “encapsulation” would be meaningless. You already use thousands of things you have no idea of how works. For instance, what is the implementation of System.String’s Clone method? I’ll give you a hint “return this;”. Now explain to me why that works, and doesn’t create problems if the same string is shared among multiple threads … 😉

99.9% of the world’s developers cannot answer the above question, without resorting to Google …

Sure, you could probably create your own custom tree view, using jQuery, PHP, CSS, JavaScript and C# etc – However, that would take you (at least) a month. It would contain thousands of lines of code in both JavaScript, CSS and C#/PHP. That month is a lot of money for your employer. Why would you want to do that, when all you need to do is to spend 5 minutes on creating 27 lines of code? Why would you spend your employer’s money basically for nothing, resulting in that your employer might go bankrupt, having to throw money literally out of the window – Resulting in that you’re eventually left without a job. Can you answer that question? In fact, let me show you your resume, and how it will look like 10 years from now for a potential employer. I should know, because I used to be the Windows API expert on the planet … 😉

  1. PhD in useless information a
  2. Masters Degree in useless information b
  3. 10 years of experience in useless information c
  4. Etc …

Do you want to know a secret? There are few differences between the art of flipping a burger, and the art of creating software. Both can be optimised to the point where MacDonalds can serve a billion burgers every day, with each employee creating an average of 100 burgers during a day. And if you choose to not use the better approach to create your software, because of vanity – Not only are you damaging your employer, but you’re also spending intellectual energy, on teaching yourself something, that is completely useless 10 years from now. Don’t believe me? Ask some WinForms developer how useful his knowledge is today …

… or a FoxPro developer for that matter … 😉

May I suggest you spend some time maybe instead learning how to play chess, or maybe the Saxophone? At least that way you’ll end up with a tangible value, that has a measurable positive impact on your life. Maybe start swimming or exercising? If you’re the average software developer, God knows you need it!

For God’s sake punk software developer, get a life!

Don’t waste your life, learning something, that nobody cares about, and that will never have any benefit for you or anybody else in the long run – At least not unless you do it in your own time, and you find it interesting and challenging …

For the record, this article, and these ideas, is probably the reason for this.

The more you fight me, and my ideas, the more you prove my point!

Epilogue – 13 minutes after I submitted a link to this article to /r/programming on Reddit, it had 75% down votes. I suspect I was the only one who upvoted it though, implying 100% of those who voted for it, voted it down. A qualified guess, is that I will soon be banned from the group. However, I’ve been wrong before. If I get banned, I will add their reasons for banning me here, which I suspect is “obsessive self promotion” – Although you can only submit links there, and the quality of this article is obviously quite high. A piece of advice when it comes to my ideas is to make up your own mind. Simply because the more “senior” your developers are, the more resistance they tend to show towards my ideas …

Software Architecture lessons from Norwegian Soccer History

Everybody who knows anything about Norwegian soccer history knows we’re like the worst soccer players on the planet individually, maybe the only ones worse are the Swedes and the Danish … 😉

Individually, it’s like we are physiologically incapable of playing the game. If you compare the average Norwegian soccer player to a Brazilian soccer player, it feels like you’re watching a Giraffe play tennis as you’re watching the Norwegian guy trying to “samba” his way through the field …

However, 25 years ago, there was this magical soccer trainer. His name was Egil “Drillo” Olsen (Google him). He broke all known soccer theory at the time, such as instead of having his players exercise on parts they were bad at, he had them exercise what they were already the best at. For instance, there was this one guy, who’s sole purpose was to shoot the ball, from one place on the field, to another place on the field. He was very good at shooting accurately and hard, but had no other skills really. Another guy’s job was to simply stay at that spot, pick down the ball from the air (his specialty), and score a goal. All other players were chosen from similar criteria, and none of these were “the best” in soccer in general. Individually, the team was arguably composed out of a bunch of “soccer retards”. However, this team humiliated many famous soccer teams, such as Brazil, Germany, England, “you name it”.

So let’s move this theory into software teams, and see if we can learn something. Let’s imagine we’re going to create a web application. OK, we know we’ll need JavaScript knowledge, so we find the best JavaScript developer in the world. We know we’ll need HTML and CSS, so we find the best CSS guy in the world. We know we’ll need C#, so we find the best C# developer in the world. However, these guys’ individual skills, become a liability to create our application. Simply because our app as a whole, doesn’t care about the brilliance of its C#, JavaScript or CSS code. Since all of these guys also are so darn special, and the most skilled in their area of expertise, they’re often also filled with vanity, and an incapacity to cut corners, in order to make their results *integrate* with the results of the guy sitting next to them. And they’ll often hold hourly long speeches about why they can’t do what they’re told to do, because of a, b or c. In fact, this problem is so common, it’s referred to as “Mythical Man Month”. If you haven’t heard of Mythical Man Month, I’ve included it below. Realise that this “theory” of software development is arguably proven scientifically, several times. It goes like this …

One man can create in on month, what two men can create in two months

Although we have known about Mythical Man Month for almost half a century, I suspect we don’t really know the reasons for it. At the least, we do NOT know how to fix it. However, I suspect Erik “Drillo” Olsen might actually have the solution for us.

The quality of your individual employee’s job is actually completely irrelevant. The only question you should be asking yourself is; “How can I integrate his results with the results of the guy sitting next to him”. This is such a universal truth, I suspect that you could literally handpick a bunch of medium skilled software developers, with the sole aim of asking how you can integrate their combined efforts, and these “mediocre” developers would far outperform a team of the same size, composed of the “best guys on the planet”. Kind of like Norway humiliated Brazil in soccer some 20 years ago …

So let’s move this on to *framework design*, which my previous article was all about. Well, the same laws applies. If you want to create a web app, you’ll need to pick a JavaScript library, so you pick the best. You’ll need to pick a CSS framework, so you pick the best. You’ll need to pick a database, so you pick the best. You’ll need to pick a server side programming language, so you pick the best. Etc, etc, etc …

AND YOU NEVER FINISH YOUR PROJECT!!!!

Because the quality of your individual parts, is completely irrelevant, because you can’t integrate these into a “whole”, because they’re built with different philosophies, architectural design patterns, goals, etc, etc, etc. And the vanity and belief in that every single library is created “perfect”, makes its maintainers incapable of perceiving anything wrong with their tools. From their point if view, it’s already *perfect*!

I suspect this is the root of all failed software projects in fact. At least intuitively it feels like a universal truth. If you instead focus on integration from DAY 1, and cut a couple of corners if necessary on quality and performance on your individual components, the end result becomes far superior, and much more easily maintained. Simply because your different individual parts doesn’t scatter “all over the place”, and focus on their individual success. They are rather focusing on the group’s success as a whole. Hence, your framework, realises it is not existing as a bunch of single entities, but rather as a collection of entities, such that group dynamics kicks in, and decides the faith for it as a whole, and hence your ability to deliver good quality end results to your clients at deadline. So hence, an inferior software development framework measured on its individual parts’ performance, will still run in circles around the combined efforts of the “best individual parts in the world”.

In fact, this also consistently improves security too, even to the point where you’ll have to cut corners, arguably from an individual point of view reducing security. The last Efail security concern about encrypted emails for instance, was not a problem with encryption standards, it was not a problem with email clients. It was in fact a problem where you could not put the blame on any single individual component in your system. Still the security flaw was as real as daylight. I happen to have this knowledge from “the best MIME library creation expert on the planet”. This was a bug in how the encryption libraries were *integrated* into the email client! So arguably, even to the point where you’ll have to “cut corner in regards to security”, security as a whole still improves, simply by realising it’s not about the individual component’s performance. It’s about “the group” as a whole (framework), and its ability to perform its task as a whole!

Mythical Man Month … What month did you say …?

Game Theory and Framework Architecture

Nash proved more than 50 years ago that if everybody competes for the same prize, we would only inflict harm, not only to ourselves, but also to the group as a whole. Or at least, this is true in a “zero sum game”, implying games where there can only be one winner. According to legend and Hollywood, his ideas were conceptualised as him and 3 friends of him were trying to pick up a beautiful blond girl at a bar. You can see this depiction below. His theories were later referred to as “Game Theory”, and they tell us something about how we can both succeed as individuals, and as a group. The paradox is that a web application framework, is actually a “group”. It is not a single entity, but rather a collection of tools, that works together the same way a “group” works together. For instance, there’s Ajax, database layers, markup generation, etc, etc, etc. All in all, resulting in that the laws of “group dynamics” are key to understand how to best implement and create a web application framework.

Phosphorus Five is a “second best web application framework”. For instance, it’s very difficult, if not flat out impossible, to re-create a social media website such as Facebook or Twitter with it. It also does not incorporate a MongoDB or Couchbase database adapter, which according to dogma are among some of the most scalable databases in the world today. Instead it contains only a MySQL database adapter, allowing you to create your own MongoDB/Couchbase wrapper yourself in C# if you wish. It does not use the latest and hottest C# features, which has the additional advantage of that it’s more easily ported to Mono, and doesn’t require you to understand all the latest features from Redmond to understand its code. Hyperlambda is slower than C#, and has less syntax than both LISP and YAML. It does not contain a rich JavaScript API. In fact, it barely contains any JavaScript at ALL! I could go through every single feature in Phosphorus Five, and illustrate how it consistently chooses the “second best” alternatives where it can. However, the end result, of its combined effort, becomes very weird due to this decision, and due to “Game Theory” and “Group Dynamics”.

For instance, with Phosphorus Five, you can create web applications without knowing any C#, JavaScript, CSS or HTML. In fact, the only thing you’ll need to learn in order to create highly rich web apps, is the difference between a colon (:) and double space (”  “). These are the two only syntactic tokens in Hyperlambda. With Phosphorus Five you can create a modal window with 5 lines of code. You can traverse your folder structure on disc in a tree view, with 25 lines of code. You can create an Ajax MySQL datagrid with 7 lines of code. Etc, etc, etc. The reasons for this, is because none of its parts are “blocking” any other parts. The “individuals in the group” becomes 100% perfectly untangled.

If I were to “micro optimise” Phosphorus Five, the same way lots of other frameworks have been micro optimised, the end result of me trying to become the winner on all parameters, would be more syntax, more complexity, more entanglement, more cognitive noise for you as you create your code, more overhead, etc, etc, etc. The individual parts of Phosphorus Five would basically “block” each other. Phosphorus Five incorporates the ideas coined by Nash more than 50 years ago in its core! And the end result, of consistently choosing to be number 2 instead of number 1, is that as a *WHOLE* it wins. Don’t believe me? Watch what a Nobel prize winner in math has to say about this …

Whenever “group dynamics” applies to your problems, you should never strife for winning the first place in any individual parts of your system. Instead, you should try to become second best on all parameters. The reasons is because as you do, the sum of your work, will inevitably be that your collective efforts becomes the winner. Nash proved this, and got the Nobel Prize for it. And I believed him, and I implemented Phosphorus Five in accordance to his theories. Resulting in … 😉

What kind of security does Phosphorus Five implement?

This seems to be a question that for weird reasons hunts me. When senior developers are out of arguments to bash Phosphorus Five about, they end up attacking it claiming that it’s insecure.

First of all, the “install.sh” file, which installs Phosphorus Five on a Linux production server, will patch and update your Ubuntu Server. This makes sure that existing security holes are eliminated, and your Linux server is up to date. Then it will install a firewall, and shut down every single network port, except port 80, port 443, and port 22. This implies that the only traffic your web server will accept, is HTTP, HTTPS and SSH.

When it has done this, it will automatically suggest for you to install an SSL key pair on your server, and suggest that you redirect all insecure traffic (port 80) to the encrypted channel (port 443). Assuming you take its advice, this makes it impossible for an adversary to see what type of data you are sending back and forth between your client and your web server. In fact, every single bit transferred over the wire will be encrypted if you choose this path. This also eliminates that a “man in the middle” can steal your credential cookie, or perform what’s known as “session highjacking”. It prevents somebody else from “impersonating” your user, pretending to be you, to gain access to your server’s data.

Then it will install the very latest stable version of Mono, which actually is not that easy, since the default Ubuntu repository contains a version that is almost 5 years old. Needless to say, but the number of security holes fixed in later versions, are probably possible to count in the hundreds, if not thousands. So this further eliminates some 100-1,000 security holes, compared to the default Ubuntu repository.

Then it stops your web server from announcing what version your server is running. This is to prevent an adversary from gaining information about which software version your server is running, such as the Apache version, Linux version, etc. Then it will turn OFF the ability to override security settings using .conf files in folders. This is a major security concern, since it in theory allows an adversary with write access to your Apache web server folder to override your web server settings. This is globally turned OFF, to prevent a whole range of security holes, that otherwise might give an adversary control over your web server.

Then it prevents the serving of “.hl” files. This is strictly not necessary, but is an additional layer of security, preventing an adversary to see your web server’s source code, trying to find holes in them, to exploit these to gain access to your data. There is a general rule in security, which is “don’t say shit”, implying the less you communicate out about your server, the less information an adversary has to start out with, in order to crack into it. If you don’t know what system the server is even running, an adversary don’t even know where to start looking for ways to penetrate it.

As it installs your SSL key pair, it will even automatically renew your keys every 90 days. This is something which is often forgotten by a web server admin, effectively rendering your web server insecure after 90 days. The above are the stuff it does to your base operating system. In addition to these steps, comes the security implementations of Phosphorus Five itself.

The passwords for your users are stored as server salted, hashed values, in a protected file. Only a “root” account has read access to this file. However, even if an adversary somehow should gain access to this file, he’ll still not be able to see its passwords, because they will appear to be rubbish. This is done by first applying a “salt” to your password, for then to “hash” the combined value, and only then store this value as the “password” in the password file. Surprisingly many developers fails this step. Then when a user logs into the system, the same salt and hashing function is applied, and the password stored on disc, is compared to this “rubbish” value. So a password such as “Thi$I4M4P@sSwo4d” might become “C90obd+yAoJ2Lgy8YiSf2VLTbI041XRaxEzNrwwej6k=”. So even if an adversary gains access to your password file, which by itself should be impossible, he still wouldn’t be able to figure out your actual passwords. This “salt” is automatically generated might I add.

In addition to the above, Phosphorus Five implements “brute force password protection”. This is useful in case some adversary has a database of commonly used passwords, and is performing what’s known as a “dictionary attack”. At which point he might have a function that tries to log in thousands of times, every second, with different passwords. Implying at some point he’ll probably “get lucky”, and successfully log into your system, at which point he knows your password. Phosphorus Five prevents this by not allowing the same username to attempt to login more than once every 20 seconds. Implying a “dictionary brute force password attack”, will basically take a decade to be successful – Even with fairly simple passwords. This poses a problem though, which implies that if you are currently experiencing a dictionary attack, you can’t login from a new client. This is fixed in Phosphorus Five, by circumventing the “brute force protection”, if you have chose to “Remember me” on some specific client – Effectively (almost) eliminating this problem for all practical concerns, since most users will probably use the same device(s) as they use Phosphorus Five, where they are encouraged to “Remember me” by default. Yet again, the cookie stored on disc on the client remembering the user, is also hashed and salted.

The cryptography libraries Phosphorus Five uses, are developed in Australia. This is actually very important, since Australia does not have export regulations on cryptography, the was US has. For instance, some parts of the cryptography applied in Phosphorus Five, would purely from a legal perspective in the US, fall into the same category as exporting nuclear weapons from the US. However, since I am based in Cyprus, and the guys who are building my cryptography libraries are based in Australia, I am legally allowed to export these cryptography libraries, to any countries not on the “terrorist list” (North Korea, Iran, etc). So a US company cannot legally even come close to the strength of the cryptography functions I happen to have in Phosphorus Five, without risking (at least in theory) the **death penalty** for being in violation of US export laws. An example of how this further strengthens the security of Phosphorus Five, is how you can for instance easily create encryption key pairs as strong as 16,000 bits! Which is 4 times the strength that the NSA is using for their own sensitive emails and communication might I add. If you don’t trust the default RNG seeding function, you can also provide your own manual salt, which is **added** to the existing salt, and does not “replace” it …

In addition to this, I have consciously chosen to NOT support cryptography functions I find suspicious myself. An extremely good example of this is S/MIME, which I knew for a fact, years ago was inferior in regards to security to PGP. Of course nobody believed me when I told them then, might I add. S/MIME and PGP are two overlapping standards, arguably doing the same – And some few weeks ago my “suspicions” were confirmed. S/MIME contains several security holes that PGP does not contain. This can be found by reading up on the Efail security holes in regards to cryptography protocols.

For synchronous encryption, I am using AES. This is the synchronous encryption protocol preferred by the NSA, which the NSA encourages American public institutions to use for “extremely volatile information”. Phosphorus Five also support 256 bits AES. I might add that AES is also an encryption protocol that has been applied on several occasions by WikiLeaks, in addition to other intelligence organisations, such as FSB and Mossad. So this is a well proven encryption protocol, that is considered “impossible to break”, by all the security paranoid organisations on the planet today.

Then comes the problem of JavaScript. It’s often easy to implement security holes in JavaScript without even knowing it. This can be done by for instance adding business logic in your JavaScript that allows an adversary to gain knowledge about your server. However, eliminating JavaScript is impossible, since this would allow you to build only websites, the way they functioned in the 1990s. However, like all attack surfaces, the objective is to reduce their size as much as possible. So the size of your JavaScript hence becomes the thing you want to control. GMail contains 1.400Kb of JavaScript. Phosphorus Five contains 5.7Kb of JavaScript. Arguably making Phosphorus Five on the JavaScript parts 245 times “more secure”. This is an oversimplification I admit, but it’s still a measuring point, allowing you to quantify your application’s “attack surface”. Phosphorus Five will by default, simply never use more than 5.7Kbof JavaScript, unless you wrap some sort of JavaScript component.

In addition to the above points, I could probably mention security details for days, without even repeating myself. And although there exists no guarantees when it comes to security, and I would of course appreciate a (*serious*) security report, reporting holes in Phosphorus Five – I can confidently assure you that I doubt you have ever seen a more secure framework on this planet than Phosphorus Five. Basically …

I have no troubles what so ever suggesting Phosphorus Five to the MI6, CIA, NSA, FSB, Mossad, WikiLeaks, etc. In fact, Phosphorus Five could probably keep your Nuclear Rockets safe!

If you do have a serious concern about parts of Phosphorus Five, in relationship to security, you can send me a report using the schema below.

Software Factory looking for Salesmen

My name is Thomas Hansen and I have spent 10 years developing something I like the refer to as “the infrastructure for a software factory”. The idea is that by using standardised modularised pre-built components, I can effectively produce software, at a rate that arguably has more in common with a “factory” than a software development company. I am now looking for forward leaning, self-driven, motivated sales staff, that can help me find customers.

The work is 100% commission based at the moment, but with a 25/75 percent profit sharing key, which might result in that you make a lot of money, for an extremely little amount of time. My claim is that I can build, produce, and deliver custom software, at least 50% less expensive than anybody else – Including Indian outsourcing companies.

I am located in Cyprus, which allows me to bid for public EU jobs, in addition to having much less overhead costs, due to the tax laws and prices in general in Cyprus. I expect my sales staff to be able to go from “zero” to a signed contract, at which point I will take over the entire process, and share 25% of all profits with whomever landed the contract. I also have a partnership with an ISP service provider, which allows me to host a multitude of servers, with an extremely well created infrastructure, and extremely high quality on the services. So even though I am a “single human being”, I have arguably everything necessary to create a “Google type of IT infrastructure”.

If you are interested in an opportunity to literally change the future with me, I encourage you to contact me at thomas@gaiasoul.com.

Here are my references.

If you are curious about what I am able to do with this tool, feel free to watch the following video.